{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10.3 Estimator介绍\n",
    "除了第三方提供的TensorFlow高层封装API，TensorFlow从1.3版本推出Datasets同时也推出了官方支持的高层封装——tf.estimator（为了引用方便，本书将tf.estimator简称为Estimator）。因为Estimator是TensorFlow官方提供的高层API，所以它更好地整合了原生态TensorFlow提供的功能。\n",
    "\n",
    "### 10.3.1 Estimator基本用法\n",
    "类似其他的高层封装，先给出在MNIST数据集上，通过Estimator实现全连接神经网络的代码："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 模型定义。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-1-6ddb2e55fa32>:9: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../../datasets/MNIST_data\\train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../../datasets/MNIST_data\\train-labels-idx1-ubyte.gz\n",
      "Extracting ../../datasets/MNIST_data\\t10k-images-idx3-ubyte.gz\n",
      "Extracting ../../datasets/MNIST_data\\t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': 'log', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x0000014E8FC3E630>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\estimator\\inputs\\queues\\feeding_queue_runner.py:62: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\estimator\\inputs\\queues\\feeding_functions.py:500: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\monitored_session.py:804: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into log\\model.ckpt.\n",
      "INFO:tensorflow:loss = 289.84402, step = 0\n",
      "INFO:tensorflow:global_step/sec: 372.998\n",
      "INFO:tensorflow:loss = 38.281395, step = 100 (0.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.7\n",
      "INFO:tensorflow:loss = 34.157536, step = 200 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.609\n",
      "INFO:tensorflow:loss = 20.971601, step = 300 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 472.148\n",
      "INFO:tensorflow:loss = 33.61521, step = 400 (0.212 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.74\n",
      "INFO:tensorflow:loss = 17.60727, step = 500 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.125\n",
      "INFO:tensorflow:loss = 14.4941435, step = 600 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.879\n",
      "INFO:tensorflow:loss = 18.659359, step = 700 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 383.629\n",
      "INFO:tensorflow:loss = 26.386875, step = 800 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 485.633\n",
      "INFO:tensorflow:loss = 13.892436, step = 900 (0.204 sec)\n",
      "INFO:tensorflow:global_step/sec: 392.806\n",
      "INFO:tensorflow:loss = 18.15664, step = 1000 (0.257 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.931\n",
      "INFO:tensorflow:loss = 10.630214, step = 1100 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 475.71\n",
      "INFO:tensorflow:loss = 14.938042, step = 1200 (0.210 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.633\n",
      "INFO:tensorflow:loss = 4.6143045, step = 1300 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 432.398\n",
      "INFO:tensorflow:loss = 2.9783978, step = 1400 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 459.745\n",
      "INFO:tensorflow:loss = 4.0549994, step = 1500 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.427\n",
      "INFO:tensorflow:loss = 3.1776118, step = 1600 (0.253 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.137\n",
      "INFO:tensorflow:loss = 5.4438343, step = 1700 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 469.568\n",
      "INFO:tensorflow:loss = 3.9517405, step = 1800 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.594\n",
      "INFO:tensorflow:loss = 8.180442, step = 1900 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.615\n",
      "INFO:tensorflow:loss = 18.638605, step = 2000 (0.234 sec)\n",
      "INFO:tensorflow:global_step/sec: 464.303\n",
      "INFO:tensorflow:loss = 4.042201, step = 2100 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 454.87\n",
      "INFO:tensorflow:loss = 4.184265, step = 2200 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 463.936\n",
      "INFO:tensorflow:loss = 2.3519099, step = 2300 (0.216 sec)\n",
      "INFO:tensorflow:global_step/sec: 377.747\n",
      "INFO:tensorflow:loss = 1.4312973, step = 2400 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.012\n",
      "INFO:tensorflow:loss = 4.1854987, step = 2500 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.656\n",
      "INFO:tensorflow:loss = 2.32231, step = 2600 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.568\n",
      "INFO:tensorflow:loss = 1.9528587, step = 2700 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 399.189\n",
      "INFO:tensorflow:loss = 5.1420717, step = 2800 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.962\n",
      "INFO:tensorflow:loss = 2.808796, step = 2900 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.585\n",
      "INFO:tensorflow:loss = 2.6663964, step = 3000 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.894\n",
      "INFO:tensorflow:loss = 3.56001, step = 3100 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.658\n",
      "INFO:tensorflow:loss = 3.6499147, step = 3200 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 357.755\n",
      "INFO:tensorflow:loss = 0.7693504, step = 3300 (0.530 sec)\n",
      "INFO:tensorflow:global_step/sec: 187.648\n",
      "INFO:tensorflow:loss = 1.1354852, step = 3400 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 454.672\n",
      "INFO:tensorflow:loss = 1.4973588, step = 3500 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.507\n",
      "INFO:tensorflow:loss = 2.400271, step = 3600 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 450.63\n",
      "INFO:tensorflow:loss = 1.6984183, step = 3700 (0.222 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.662\n",
      "INFO:tensorflow:loss = 1.0660267, step = 3800 (0.225 sec)\n",
      "INFO:tensorflow:global_step/sec: 457.395\n",
      "INFO:tensorflow:loss = 1.1374506, step = 3900 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.885\n",
      "INFO:tensorflow:loss = 0.97266185, step = 4000 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 451.241\n",
      "INFO:tensorflow:loss = 1.2124919, step = 4100 (0.220 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.723\n",
      "INFO:tensorflow:loss = 0.6535756, step = 4200 (0.217 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 403.508\n",
      "INFO:tensorflow:loss = 0.7413038, step = 4300 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 503.526\n",
      "INFO:tensorflow:loss = 0.332745, step = 4400 (0.211 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.554\n",
      "INFO:tensorflow:loss = 0.71886796, step = 4500 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.534\n",
      "INFO:tensorflow:loss = 0.33426118, step = 4600 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.092\n",
      "INFO:tensorflow:loss = 0.64932704, step = 4700 (0.266 sec)\n",
      "INFO:tensorflow:global_step/sec: 371.362\n",
      "INFO:tensorflow:loss = 0.2140636, step = 4800 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.165\n",
      "INFO:tensorflow:loss = 0.3477956, step = 4900 (0.276 sec)\n",
      "INFO:tensorflow:global_step/sec: 372.071\n",
      "INFO:tensorflow:loss = 0.4882283, step = 5000 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 335.74\n",
      "INFO:tensorflow:loss = 0.108943395, step = 5100 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.492\n",
      "INFO:tensorflow:loss = 0.30899796, step = 5200 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.323\n",
      "INFO:tensorflow:loss = 0.2539618, step = 5300 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.696\n",
      "INFO:tensorflow:loss = 0.69746876, step = 5400 (0.295 sec)\n",
      "INFO:tensorflow:global_step/sec: 422.829\n",
      "INFO:tensorflow:loss = 0.9973472, step = 5500 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 423.121\n",
      "INFO:tensorflow:loss = 0.5158064, step = 5600 (0.236 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.438\n",
      "INFO:tensorflow:loss = 0.21823362, step = 5700 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 352.608\n",
      "INFO:tensorflow:loss = 2.1042302, step = 5800 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 424.06\n",
      "INFO:tensorflow:loss = 2.8646166, step = 5900 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 380.201\n",
      "INFO:tensorflow:loss = 0.70516586, step = 6000 (0.263 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.136\n",
      "INFO:tensorflow:loss = 0.93671143, step = 6100 (0.237 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.48\n",
      "INFO:tensorflow:loss = 0.9132229, step = 6200 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 376.704\n",
      "INFO:tensorflow:loss = 0.1501778, step = 6300 (0.265 sec)\n",
      "INFO:tensorflow:global_step/sec: 354.088\n",
      "INFO:tensorflow:loss = 0.5205095, step = 6400 (0.282 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.345\n",
      "INFO:tensorflow:loss = 0.10736393, step = 6500 (0.268 sec)\n",
      "INFO:tensorflow:global_step/sec: 353.932\n",
      "INFO:tensorflow:loss = 0.36842126, step = 6600 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 356.384\n",
      "INFO:tensorflow:loss = 0.055623155, step = 6700 (0.281 sec)\n",
      "INFO:tensorflow:global_step/sec: 395.916\n",
      "INFO:tensorflow:loss = 0.3800242, step = 6800 (0.255 sec)\n",
      "INFO:tensorflow:global_step/sec: 375.533\n",
      "INFO:tensorflow:loss = 0.24424417, step = 6900 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 390.147\n",
      "INFO:tensorflow:loss = 0.04620771, step = 7000 (0.256 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.255\n",
      "INFO:tensorflow:loss = 0.11668793, step = 7100 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 370.172\n",
      "INFO:tensorflow:loss = 0.39854932, step = 7200 (0.271 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.21\n",
      "INFO:tensorflow:loss = 1.3530949, step = 7300 (0.291 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.887\n",
      "INFO:tensorflow:loss = 0.379996, step = 7400 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 368.632\n",
      "INFO:tensorflow:loss = 0.2487037, step = 7500 (0.270 sec)\n",
      "INFO:tensorflow:global_step/sec: 439.77\n",
      "INFO:tensorflow:loss = 0.73983353, step = 7600 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.315\n",
      "INFO:tensorflow:loss = 0.28550637, step = 7700 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 453.697\n",
      "INFO:tensorflow:loss = 1.0074421, step = 7800 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 443.751\n",
      "INFO:tensorflow:loss = 0.3774007, step = 7900 (0.238 sec)\n",
      "INFO:tensorflow:global_step/sec: 391.088\n",
      "INFO:tensorflow:loss = 0.27814937, step = 8000 (0.240 sec)\n",
      "INFO:tensorflow:global_step/sec: 427.01\n",
      "INFO:tensorflow:loss = 0.13215728, step = 8100 (0.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 461.066\n",
      "INFO:tensorflow:loss = 1.5253836, step = 8200 (0.201 sec)\n",
      "INFO:tensorflow:global_step/sec: 430.906\n",
      "INFO:tensorflow:loss = 0.1800654, step = 8300 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 425.344\n",
      "INFO:tensorflow:loss = 0.17152911, step = 8400 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.087\n",
      "INFO:tensorflow:loss = 2.5224621, step = 8500 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 436.271\n",
      "INFO:tensorflow:loss = 0.33390447, step = 8600 (0.229 sec)\n",
      "INFO:tensorflow:global_step/sec: 446.498\n",
      "INFO:tensorflow:loss = 0.009942287, step = 8700 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 409.364\n",
      "INFO:tensorflow:loss = 0.015857715, step = 8800 (0.244 sec)\n",
      "INFO:tensorflow:global_step/sec: 496.351\n",
      "INFO:tensorflow:loss = 0.047282595, step = 8900 (0.201 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.556\n",
      "INFO:tensorflow:loss = 0.020891212, step = 9000 (0.223 sec)\n",
      "INFO:tensorflow:global_step/sec: 445.78\n",
      "INFO:tensorflow:loss = 0.011128712, step = 9100 (0.224 sec)\n",
      "INFO:tensorflow:global_step/sec: 429.003\n",
      "INFO:tensorflow:loss = 0.028417539, step = 9200 (0.233 sec)\n",
      "INFO:tensorflow:global_step/sec: 458.412\n",
      "INFO:tensorflow:loss = 0.13105117, step = 9300 (0.218 sec)\n",
      "INFO:tensorflow:global_step/sec: 465.061\n",
      "INFO:tensorflow:loss = 0.033948757, step = 9400 (0.215 sec)\n",
      "INFO:tensorflow:global_step/sec: 460.555\n",
      "INFO:tensorflow:loss = 0.03479446, step = 9500 (0.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 456.647\n",
      "INFO:tensorflow:loss = 0.06924718, step = 9600 (0.219 sec)\n",
      "INFO:tensorflow:global_step/sec: 431.014\n",
      "INFO:tensorflow:loss = 0.08460676, step = 9700 (0.232 sec)\n",
      "INFO:tensorflow:global_step/sec: 441.38\n",
      "INFO:tensorflow:loss = 0.8263117, step = 9800 (0.227 sec)\n",
      "INFO:tensorflow:global_step/sec: 447.361\n",
      "INFO:tensorflow:loss = 1.5937722, step = 9900 (0.224 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10000 into log\\model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 9.191972.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-21-09:50:42\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from log\\model.ckpt-10000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-21-09:50:43\n",
      "INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.9806, average_loss = 0.09410643, global_step = 10000, loss = 11.912206\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: log\\model.ckpt-10000\n",
      "\n",
      "test_results:  {'accuracy': 0.9806, 'average_loss': 0.09410643, 'loss': 11.912206, 'global_step': 10000}\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "# 1. 模型定义\n",
    "# 将TensorFlow日志信息输出到屏幕\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "\n",
    "mnist = input_data.read_data_sets(\"../../datasets/MNIST_data\", one_hot=False)\n",
    "\n",
    "# 指定神经网络的输入层。这里所有指定的输入层都会拼接在一起作为整个神经网络的输入\n",
    "feature_columns = [tf.feature_column.numeric_column(\"image\", shape=[784])]\n",
    "\n",
    "# 通过TensorFlow提供的封装好的Estimator定义神经网络模型。\n",
    "#   feature_columns参数给出了神经网络输入层需要用到的数据；\n",
    "#   hidden_units参数给出了神经网络的结构（注意这DNNClassifier只能定义多层全连接层\n",
    "#               神经网络，而hidden_units列表中给出了每一层隐藏层的节点个数）；\n",
    "#   n_classes参数给出了总共类目的数量；\n",
    "#   optimizer参数给出了使用的优化函数；\n",
    "#   model_dir参数给出了训练过程中的loss及其他指标的保存地址，可使用TensorBoard可视化\n",
    "estimator = tf.estimator.DNNClassifier(feature_columns=feature_columns,\n",
    "                                       hidden_units=[500],\n",
    "                                       n_classes=10,\n",
    "                                       optimizer=tf.train.AdamOptimizer(),\n",
    "                                       model_dir=\"log\")\n",
    "\n",
    "\n",
    "# 2. 训练模型\n",
    "# 先定义输入数据：\n",
    "#   x:如果上面feature_columns中指定了多个，那么这里需要对每一个对应提供数据\n",
    "#   y:需要提供每一个x对应的正确答案，这里要求分类的结果是一个正数\n",
    "#   num_epochs:指定数据循环轮数，测试时为1\n",
    "#   batch_size:指定一个batch大小\n",
    "#   shuffle:指定是否需要对数据进行随机打乱\n",
    "train_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "      x={\"image\": mnist.train.images},\n",
    "      y=mnist.train.labels.astype(np.int32),\n",
    "      num_epochs=None,\n",
    "      batch_size=128,\n",
    "      shuffle=True)\n",
    "\n",
    "# 注意这里没有指定损失函数，通过DNNClassifier定义的模型会使用交叉熵作为损失函数\n",
    "estimator.train(input_fn=train_input_fn, steps=10000)\n",
    "\n",
    "\n",
    "# 3. 测试模型\n",
    "# 定义测试时的数据输入，和训练时的数据输入基本一致\n",
    "test_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "      x={\"image\": mnist.test.images},\n",
    "      y=mnist.test.labels.astype(np.int32),\n",
    "      num_epochs=1,\n",
    "      batch_size=128,\n",
    "      shuffle=False)\n",
    "\n",
    "test_results = estimator.evaluate(input_fn=test_input_fn)\n",
    "print('\\ntest_results: ',test_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从以上代码可以看出，使用预先定义好的**Estimator可以更加深层次地封装神经网络的定义和训练过程。在这个过程中，用户只需要关注模型的输入以及模型的结构，其他的工作都可以通过Estimator自动完成。**下图是使用TensorBoard来可视化训练过程（第11章会介绍TensorBoard）：\n",
    "<p align='center'>\n",
    "    <img src=images/图10.3.JPG>\n",
    "</p>\n",
    "\n",
    "### 10.3.2 Estimator自定义模型\n",
    "但是**预先定义好的Estimator功能有限**，比如：\n",
    "- 不能灵活选择模型的结构，如卷积神经网络或者循环神经网络；\n",
    "- 也没有办法支持自定义的损失函数；\n",
    "- 每一层使用的激活函数也是自定义好的。\n",
    "\n",
    "所以为了更好地使用Estimator，这一小节将介绍如何使用Estimator自定义模型。下面代码展示了如何通过自定义的方式使用CNN解决MNIST问题："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': 'log', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001865F5EE8D0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
      "WARNING:tensorflow:From <ipython-input-1-851d9bfeff4a>:76: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../../datasets/MNIST_data\\train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../../datasets/MNIST_data\\train-labels-idx1-ubyte.gz\n",
      "Extracting ../../datasets/MNIST_data\\t10k-images-idx3-ubyte.gz\n",
      "Extracting ../../datasets/MNIST_data\\t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\estimator\\inputs\\queues\\feeding_queue_runner.py:62: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\estimator\\inputs\\queues\\feeding_functions.py:500: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "WARNING:tensorflow:From d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\monitored_session.py:804: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "To construct input pipelines, use the `tf.data` module.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into log\\model.ckpt.\n",
      "INFO:tensorflow:loss = 2.3109174, step = 0\n",
      "INFO:tensorflow:global_step/sec: 198.054\n",
      "INFO:tensorflow:loss = 2.100204, step = 100 (0.507 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.445\n",
      "INFO:tensorflow:loss = 0.9067886, step = 200 (0.434 sec)\n",
      "INFO:tensorflow:global_step/sec: 225.729\n",
      "INFO:tensorflow:loss = 0.5804382, step = 300 (0.443 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.568\n",
      "INFO:tensorflow:loss = 0.3522097, step = 400 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.944\n",
      "INFO:tensorflow:loss = 0.42299473, step = 500 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.676\n",
      "INFO:tensorflow:loss = 0.27530017, step = 600 (0.430 sec)\n",
      "INFO:tensorflow:global_step/sec: 246.157\n",
      "INFO:tensorflow:loss = 0.26454964, step = 700 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 227.249\n",
      "INFO:tensorflow:loss = 0.14974761, step = 800 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.645\n",
      "INFO:tensorflow:loss = 0.3270427, step = 900 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.374\n",
      "INFO:tensorflow:loss = 0.2034138, step = 1000 (0.415 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.165\n",
      "INFO:tensorflow:loss = 0.16604523, step = 1100 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.353\n",
      "INFO:tensorflow:loss = 0.24170524, step = 1200 (0.416 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.128\n",
      "INFO:tensorflow:loss = 0.08574037, step = 1300 (0.418 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.544\n",
      "INFO:tensorflow:loss = 0.19617705, step = 1400 (0.416 sec)\n",
      "INFO:tensorflow:global_step/sec: 231.39\n",
      "INFO:tensorflow:loss = 0.18798581, step = 1500 (0.432 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.45\n",
      "INFO:tensorflow:loss = 0.3361919, step = 1600 (0.421 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.849\n",
      "INFO:tensorflow:loss = 0.107345715, step = 1700 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.71\n",
      "INFO:tensorflow:loss = 0.19255824, step = 1800 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.899\n",
      "INFO:tensorflow:loss = 0.16904907, step = 1900 (0.429 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.261\n",
      "INFO:tensorflow:loss = 0.11622687, step = 2000 (0.421 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.543\n",
      "INFO:tensorflow:loss = 0.14788911, step = 2100 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 226.65\n",
      "INFO:tensorflow:loss = 0.118271306, step = 2200 (0.441 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.854\n",
      "INFO:tensorflow:loss = 0.1057172, step = 2300 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.38\n",
      "INFO:tensorflow:loss = 0.054488383, step = 2400 (0.429 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.221\n",
      "INFO:tensorflow:loss = 0.14665069, step = 2500 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.981\n",
      "INFO:tensorflow:loss = 0.11223309, step = 2600 (0.413 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.055\n",
      "INFO:tensorflow:loss = 0.14912297, step = 2700 (0.427 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.013\n",
      "INFO:tensorflow:loss = 0.109044984, step = 2800 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 241.122\n",
      "INFO:tensorflow:loss = 0.09235464, step = 2900 (0.415 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.227\n",
      "INFO:tensorflow:loss = 0.046369832, step = 3000 (0.427 sec)\n",
      "INFO:tensorflow:global_step/sec: 243.862\n",
      "INFO:tensorflow:loss = 0.11126307, step = 3100 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.777\n",
      "INFO:tensorflow:loss = 0.06992188, step = 3200 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.362\n",
      "INFO:tensorflow:loss = 0.14455044, step = 3300 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.094\n",
      "INFO:tensorflow:loss = 0.06927288, step = 3400 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.471\n",
      "INFO:tensorflow:loss = 0.085858695, step = 3500 (0.418 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.123\n",
      "INFO:tensorflow:loss = 0.121040046, step = 3600 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 230.409\n",
      "INFO:tensorflow:loss = 0.09017362, step = 3700 (0.436 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.27\n",
      "INFO:tensorflow:loss = 0.10883325, step = 3800 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.751\n",
      "INFO:tensorflow:loss = 0.14511023, step = 3900 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.158\n",
      "INFO:tensorflow:loss = 0.1015262, step = 4000 (0.438 sec)\n",
      "INFO:tensorflow:global_step/sec: 219.834\n",
      "INFO:tensorflow:loss = 0.10037376, step = 4100 (0.453 sec)\n",
      "INFO:tensorflow:global_step/sec: 221.58\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.11122318, step = 4200 (0.451 sec)\n",
      "INFO:tensorflow:global_step/sec: 231.555\n",
      "INFO:tensorflow:loss = 0.06660571, step = 4300 (0.432 sec)\n",
      "INFO:tensorflow:global_step/sec: 224.608\n",
      "INFO:tensorflow:loss = 0.12405339, step = 4400 (0.445 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.614\n",
      "INFO:tensorflow:loss = 0.071572065, step = 4500 (0.437 sec)\n",
      "INFO:tensorflow:global_step/sec: 230.431\n",
      "INFO:tensorflow:loss = 0.07183095, step = 4600 (0.450 sec)\n",
      "INFO:tensorflow:global_step/sec: 223.86\n",
      "INFO:tensorflow:loss = 0.07558228, step = 4700 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.25\n",
      "INFO:tensorflow:loss = 0.14471331, step = 4800 (0.436 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.759\n",
      "INFO:tensorflow:loss = 0.15066326, step = 4900 (0.453 sec)\n",
      "INFO:tensorflow:global_step/sec: 223.847\n",
      "INFO:tensorflow:loss = 0.048634168, step = 5000 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.765\n",
      "INFO:tensorflow:loss = 0.11927282, step = 5100 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.847\n",
      "INFO:tensorflow:loss = 0.04964294, step = 5200 (0.435 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.221\n",
      "INFO:tensorflow:loss = 0.047458604, step = 5300 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.622\n",
      "INFO:tensorflow:loss = 0.03636572, step = 5400 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 230.17\n",
      "INFO:tensorflow:loss = 0.033165522, step = 5500 (0.434 sec)\n",
      "INFO:tensorflow:global_step/sec: 240.873\n",
      "INFO:tensorflow:loss = 0.05917631, step = 5600 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 223.182\n",
      "INFO:tensorflow:loss = 0.036055822, step = 5700 (0.432 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.947\n",
      "INFO:tensorflow:loss = 0.06765696, step = 5800 (0.419 sec)\n",
      "INFO:tensorflow:global_step/sec: 229.008\n",
      "INFO:tensorflow:loss = 0.10138157, step = 5900 (0.437 sec)\n",
      "INFO:tensorflow:global_step/sec: 241.636\n",
      "INFO:tensorflow:loss = 0.0599567, step = 6000 (0.414 sec)\n",
      "INFO:tensorflow:global_step/sec: 231.48\n",
      "INFO:tensorflow:loss = 0.10607214, step = 6100 (0.448 sec)\n",
      "INFO:tensorflow:global_step/sec: 230.637\n",
      "INFO:tensorflow:loss = 0.047390576, step = 6200 (0.418 sec)\n",
      "INFO:tensorflow:global_step/sec: 210.468\n",
      "INFO:tensorflow:loss = 0.018824762, step = 6300 (0.477 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.58\n",
      "INFO:tensorflow:loss = 0.03113209, step = 6400 (0.421 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.81\n",
      "INFO:tensorflow:loss = 0.013895448, step = 6500 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.544\n",
      "INFO:tensorflow:loss = 0.13936457, step = 6600 (0.438 sec)\n",
      "INFO:tensorflow:global_step/sec: 224.568\n",
      "INFO:tensorflow:loss = 0.019445322, step = 6700 (0.448 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.996\n",
      "INFO:tensorflow:loss = 0.12506914, step = 6800 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 243.686\n",
      "INFO:tensorflow:loss = 0.07159779, step = 6900 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.37\n",
      "INFO:tensorflow:loss = 0.038653363, step = 7000 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 231.961\n",
      "INFO:tensorflow:loss = 0.09177716, step = 7100 (0.433 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.376\n",
      "INFO:tensorflow:loss = 0.08647606, step = 7200 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 228.204\n",
      "INFO:tensorflow:loss = 0.03711696, step = 7300 (0.438 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.809\n",
      "INFO:tensorflow:loss = 0.051995378, step = 7400 (0.417 sec)\n",
      "INFO:tensorflow:global_step/sec: 239.381\n",
      "INFO:tensorflow:loss = 0.07990403, step = 7500 (0.418 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.444\n",
      "INFO:tensorflow:loss = 0.0946942, step = 7600 (0.428 sec)\n",
      "INFO:tensorflow:global_step/sec: 225.973\n",
      "INFO:tensorflow:loss = 0.050493322, step = 7700 (0.445 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.388\n",
      "INFO:tensorflow:loss = 0.026839383, step = 7800 (0.419 sec)\n",
      "INFO:tensorflow:global_step/sec: 238.636\n",
      "INFO:tensorflow:loss = 0.08596471, step = 7900 (0.419 sec)\n",
      "INFO:tensorflow:global_step/sec: 226.826\n",
      "INFO:tensorflow:loss = 0.090162665, step = 8000 (0.443 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.19\n",
      "INFO:tensorflow:loss = 0.17700203, step = 8100 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.768\n",
      "INFO:tensorflow:loss = 0.08353189, step = 8200 (0.421 sec)\n",
      "INFO:tensorflow:global_step/sec: 232.147\n",
      "INFO:tensorflow:loss = 0.090236254, step = 8300 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 236.905\n",
      "INFO:tensorflow:loss = 0.037523177, step = 8400 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.967\n",
      "INFO:tensorflow:loss = 0.12067883, step = 8500 (0.424 sec)\n",
      "INFO:tensorflow:global_step/sec: 245.546\n",
      "INFO:tensorflow:loss = 0.06962856, step = 8600 (0.423 sec)\n",
      "INFO:tensorflow:global_step/sec: 227.509\n",
      "INFO:tensorflow:loss = 0.065262094, step = 8700 (0.425 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.125\n",
      "INFO:tensorflow:loss = 0.032266643, step = 8800 (0.431 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.35\n",
      "INFO:tensorflow:loss = 0.049534827, step = 8900 (0.429 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.095\n",
      "INFO:tensorflow:loss = 0.04411458, step = 9000 (0.426 sec)\n",
      "INFO:tensorflow:global_step/sec: 244.404\n",
      "INFO:tensorflow:loss = 0.061221246, step = 9100 (0.407 sec)\n",
      "INFO:tensorflow:global_step/sec: 227.855\n",
      "INFO:tensorflow:loss = 0.027412934, step = 9200 (0.441 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.485\n",
      "INFO:tensorflow:loss = 0.08632137, step = 9300 (0.425 sec)\n",
      "INFO:tensorflow:global_step/sec: 233.467\n",
      "INFO:tensorflow:loss = 0.06363426, step = 9400 (0.428 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.822\n",
      "INFO:tensorflow:loss = 0.038599543, step = 9500 (0.422 sec)\n",
      "INFO:tensorflow:global_step/sec: 234.859\n",
      "INFO:tensorflow:loss = 0.025254607, step = 9600 (0.428 sec)\n",
      "INFO:tensorflow:global_step/sec: 237.43\n",
      "INFO:tensorflow:loss = 0.040909953, step = 9700 (0.420 sec)\n",
      "INFO:tensorflow:global_step/sec: 230.823\n",
      "INFO:tensorflow:loss = 0.014260523, step = 9800 (0.433 sec)\n",
      "INFO:tensorflow:global_step/sec: 235.267\n",
      "INFO:tensorflow:loss = 0.038355615, step = 9900 (0.425 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10000 into log\\model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.04029271.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-21-11:33:17\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from log\\model.ckpt-10000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-21-11:33:17\n",
      "INFO:tensorflow:Saving dict for global step 10000: accuracy = 0.9856, global_step = 10000, loss = 0.043183208\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: log\\model.ckpt-10000\n",
      "\n",
      "test_results:  {'accuracy': 0.9856, 'loss': 0.043183208, 'global_step': 10000} \n",
      "\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from log\\model.ckpt-10000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADa9JREFUeJzt3X2MXPV1xvHnib1e4jW0OMTGNQYnhKA4NJBqYxK5rRxRp9AEmSiBYqmWK6UsakGCKmqLLEVBaptSFEJpk0ZyihsT8ZYGKFbipkFWW4pKHS+Id9NCqUtcb72AaW0C+AWf/rHX0QZ2fjvM2531+X4ka2buuXfu0fU+e2f2N3d+jggByOcddTcAoB6EH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUrN7ubM5HozjNNTLXQKpvK4f62AccDPrthV+2+dLuknSLEl/FRHXldY/TkM61+e1s0sABdtia9Prtvyy3/YsSV+TdIGkZZLW2F7W6vMB6K123vMvl/RsRDwXEQcl3SFpdWfaAtBt7YR/saQfTXq8q1r2U2yP2B61PXpIB9rYHYBOaif8U/1R4S3XB0fEhogYjojhAQ22sTsAndRO+HdJWjLp8SmSdrfXDoBeaSf82yWdYfs9tudIulTS5s60BaDbWh7qi4jDtq+U9PeaGOrbGBFPdqwzAF3V1jh/RGyRtKVDvQDoIT7eCyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFJtzdJre6ek/ZLekHQ4IoY70RSA7msr/JWPR8SLHXgeAD3Ey34gqXbDH5J+YPsh2yOdaAhAb7T7sn9FROy2vUDSfbafjoj7J69Q/VIYkaTjNLfN3QHolLbO/BGxu7odl3SPpOVTrLMhIoYjYnhAg+3sDkAHtRx+20O2jz96X9InJD3RqcYAdFc7L/sXSrrH9tHnuS0ivt+RrgB0Xcvhj4jnJJ3dwV4A9BBDfUBShB9IivADSRF+ICnCDyRF+IGkOnFVXwovXfaxhrVT1z5b3Pbp8YXF+sEDA8X64tvL9bm7XmlYO/LIU8VtkRdnfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IinH+Jv3+793WsPaZoZfLG5/e5s5Xlss7D7/asHbTCx9vc+cz1w/HT2tYG7rhZ4rbzt76UKfb6Tuc+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKUdEz3Z2gufHuT6vZ/vrpB9/9tyGtRc/VP4deuKO8jF++QMu1ud86H+L9evPurthbdU7Xytu+71X5xXrn5zb+LsC2vVaHCzWtx0YKtZXHneo5X2/73uXF+vvH9ne8nPXaVts1b7YW/6BqnDmB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkpr2e3/ZGSZ+SNB4RZ1XL5ku6U9JSSTslXRIR01zUPrMNfWdbodbec5/Q3ub6i5NXNqz90Yql5X3/U3nOgetXvq+Fjpoz+7UjxfrQY2PF+rvuv6tY//k5jec7mLuzPBdCBs2c+b8p6fw3LbtG0taIOEPS1uoxgBlk2vBHxP2S9r5p8WpJm6r7myRd1OG+AHRZq+/5F0bEmCRVtws61xKAXuj6d/jZHpE0IknHaW63dwegSa2e+ffYXiRJ1e14oxUjYkNEDEfE8IAGW9wdgE5rNfybJa2r7q+TdG9n2gHQK9OG3/btkh6UdKbtXbY/J+k6SatsPyNpVfUYwAwy7Xv+iFjToDQzL8w/Bh3+nz0Na0N3Na5J0hvTPPfQd15qoaPO2PNbHyvWPzin/OP75b1nNqwt/evnitseLlaPDXzCD0iK8ANJEX4gKcIPJEX4gaQIP5AUU3SjNrNPW1Ksf3X9V4v1Ac8q1v/mpl9pWHvX2IPFbTPgzA8kRfiBpAg/kBThB5Ii/EBShB9IivADSTHOj9o8/buLi/WPDJZnmn7yYHn68flPvfq2e8qEMz+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJMU4P7rqwCc/0rD28GdvnGbr8gxPv33VVcX6O//lh9M8f26c+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gqWnH+W1vlPQpSeMRcVa17FpJl0l6oVptfURs6VaTmLmev6Dx+WWey+P4a/5zVbE+9/uPFutRrKKZM/83JZ0/xfIbI+Kc6h/BB2aYacMfEfdL2tuDXgD0UDvv+a+0/ZjtjbZP7FhHAHqi1fB/XdLpks6RNCbphkYr2h6xPWp79JAOtLg7AJ3WUvgjYk9EvBERRyR9Q9LywrobImI4IoYHprlQA0DvtBR+24smPfy0pCc60w6AXmlmqO92SSslnWR7l6QvSlpp+xxNjKbslHR5F3sE0AXThj8i1kyx+OYu9IIZ6B3HH1+sr/2lBxrW9h15vbjt+JfeW6wPHtherKOMT/gBSRF+ICnCDyRF+IGkCD+QFOEHkuKru9GWZ679YLH+3ZP+smFt9TOfKW47uIWhvG7izA8kRfiBpAg/kBThB5Ii/EBShB9IivADSTHOj6L/+42PFuuP/fqfF+v/cfhQw9orf3pKcdtBjRXraA9nfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IinH+5GYv/rli/eov3FmsD7r8I3Tpo2sb1t79d1yvXyfO/EBShB9IivADSRF+ICnCDyRF+IGkCD+Q1LTj/LaXSLpF0smSjkjaEBE32Z4v6U5JSyXtlHRJRLzcvVbRCs8u/xef/d1dxfrF814q1m/dv6BYX/iFxueXI8Ut0W3NnPkPS/p8RHxA0kclXWF7maRrJG2NiDMkba0eA5ghpg1/RIxFxMPV/f2SdkhaLGm1pE3VapskXdStJgF03tt6z297qaQPS9omaWFEjEkTvyAklV//AegrTYff9jxJd0m6OiL2vY3tRmyP2h49pAOt9AigC5oKv+0BTQT/1oi4u1q8x/aiqr5I0vhU20bEhogYjojhAQ12omcAHTBt+G1b0s2SdkTEVyaVNktaV91fJ+nezrcHoFuauaR3haS1kh63/Ui1bL2k6yR92/bnJD0v6eLutIi2nH1msfyHC77V1tN/7Uvl//afffTBtp4f3TNt+CPiAUluUD6vs+0A6BU+4QckRfiBpAg/kBThB5Ii/EBShB9Iiq/uPgbMWvb+hrWRO9r77NWyjVcU60u/9a9tPT/qw5kfSIrwA0kRfiApwg8kRfiBpAg/kBThB5JinP8Y8PTvnNiwduHcpr9xbUqn/OPB8goRbT0/6sOZH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSYpx/Bnj9wuXF+tYLbyhU53a2GRwzOPMDSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFLTjvPbXiLpFkknSzoiaUNE3GT7WkmXSXqhWnV9RGzpVqOZ7V4xq1g/dXbrY/m37l9QrA/sK1/Pz9X8M1czH/I5LOnzEfGw7eMlPWT7vqp2Y0R8uXvtAeiWacMfEWOSxqr7+23vkLS4240B6K639Z7f9lJJH5a0rVp0pe3HbG+0PeV3SdkesT1qe/SQDrTVLIDOaTr8tudJukvS1RGxT9LXJZ0u6RxNvDKY8gPmEbEhIoYjYnhAgx1oGUAnNBV+2wOaCP6tEXG3JEXEnoh4IyKOSPqGpPLVJwD6yrTht21JN0vaERFfmbR80aTVPi3pic63B6Bbmvlr/wpJayU9bvuRatl6SWtsn6OJ0Z6dki7vSodoy5+8tKxYf/BXlxbrMfZ4B7tBP2nmr/0PSPIUJcb0gRmMT/gBSRF+ICnCDyRF+IGkCD+QFOEHknL0cIrlEzw/zvV5PdsfkM222Kp9sXeqofm34MwPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0n1dJzf9guS/mvSopMkvdizBt6efu2tX/uS6K1VnezttIh4dzMr9jT8b9m5PRoRw7U1UNCvvfVrXxK9taqu3njZDyRF+IGk6g7/hpr3X9KvvfVrXxK9taqW3mp9zw+gPnWf+QHUpJbw2z7f9r/Zftb2NXX00IjtnbYft/2I7dGae9loe9z2E5OWzbd9n+1nqtspp0mrqbdrbf93dewesf1rNfW2xPY/2N5h+0nbV1XLaz12hb5qOW49f9lve5akf5e0StIuSdslrYmIp3raSAO2d0oajojax4Rt/7KkVyTdEhFnVcuul7Q3Iq6rfnGeGBF/0Ce9XSvplbpnbq4mlFk0eWZpSRdJ+k3VeOwKfV2iGo5bHWf+5ZKejYjnIuKgpDskra6hj74XEfdL2vumxaslbarub9LED0/PNeitL0TEWEQ8XN3fL+nozNK1HrtCX7WoI/yLJf1o0uNd6q8pv0PSD2w/ZHuk7mamsLCaNv3o9OkLau7nzaadubmX3jSzdN8cu1ZmvO60OsI/1VcM9dOQw4qI+AVJF0i6onp5i+Y0NXNzr0wxs3RfaHXG606rI/y7JC2Z9PgUSbtr6GNKEbG7uh2XdI/6b/bhPUcnSa1ux2vu5yf6aebmqWaWVh8cu36a8bqO8G+XdIbt99ieI+lSSZtr6OMtbA9Vf4iR7SFJn1D/zT68WdK66v46SffW2MtP6ZeZmxvNLK2aj12/zXhdy4d8qqGMP5M0S9LGiPjjnjcxBdvv1cTZXpqYxPS2OnuzfbuklZq46muPpC9K+ltJ35Z0qqTnJV0cET3/w1uD3lZq4qXrT2ZuPvoeu8e9/aKkf5b0uKQj1eL1mnh/XduxK/S1RjUcNz7hByTFJ/yApAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyT1//RJwTziTb07AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 7\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADftJREFUeJzt3X+MXXWZx/HP0zJtsVRo01K7pVKK7UJhQ9FJFdFdCIuLxFhMFtZm1x2M7rhZ2dWkiZJmEzGKIUZAN2vcVGksCT9k+VkjKrVqAHdSOmVZWqnaLjuLtZMOTUdbdLftTB//mFMytnO+9/be8+NOn/crae695zn3nCcXPnPuvd9zz9fcXQDimVJ3AwDqQfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwR1RpU7m2bTfYZmVrlLIJT/1291xA9bM+u2FX4zu07SVyRNlfQNd78jtf4MzdTb7Zp2dgkgYYtvbnrdlt/2m9lUSV+V9F5JyyWtNrPlrW4PQLXa+cy/UtJud3/Z3Y9IelDSqmLaAlC2dsK/UNIvxz3eky37A2bWa2b9ZtZ/VIfb2B2AIrUT/om+VDjp98Huvs7du929u0vT29gdgCK1E/49khaNe3yepL3ttQOgKu2Ef6ukpWZ2gZlNk/RBSRuLaQtA2Voe6nP3ETO7RdL3NTbUt97df1pYZwBK1dY4v7s/KenJgnoBUCFO7wWCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+IKhKL92N1gx8/opkfXTGSRdQet28S15NPrfvskda6um4C3/44WR91nNn5tbm/8t/tLVvtIcjPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExTh/Bxj+ztJkfceKfy1t30fzTxFoys+u/kayfl/3gtzaQ5v+LPnc0Z27WuoJzeHIDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBtTXOb2YDkg5JGpU04u7dRTR1umk0jv+TFQ+Wtu9/+/WSZP2uvmuT9cXnp68H8NTyR5P1v541mFu7/ea5yecu+TTj/GUq4iSfq919fwHbAVAh3vYDQbUbfpf0lJltM7PeIhoCUI123/Zf6e57zexcSZvM7Gfu/vT4FbI/Cr2SNENvaHN3AIrS1pHf3fdmt0OSHpO0coJ11rl7t7t3d2l6O7sDUKCWw29mM81s1vH7kt4jaUdRjQEoVztv++dLeszMjm/nfnf/XiFdAShdy+F395clXVZgL5PWyDVvS9Z/eNlXG2yhK1n98vCyZP1Hf5U4vWLvUPK5y4b7k/UpM2Yk61/Y8ifJ+tq523NrI7NHks9FuRjqA4Ii/EBQhB8IivADQRF+ICjCDwTFpbsL8NrCacn6lAZ/YxsN5f34/enhtNGXf56st2P3Zy9P1u+fc2eDLeSf1Xne9zj21IlXHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpy/AOfc25es/2X/3yTrNnwwWR8ZHDjFjorz0et/kKyfNYWrM01WHPmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjG+Ssw+tIv6m4h18DtVyTrHznnSw22kL6095rBd+TWZv1gZ/K5ow32jPZw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBqO85vZeknvkzTk7pdmy+ZI+pakxZIGJN3k7sPltYlW/fpD6XH8n/xtehz/7Cnpcfy+w1OT9Rc+n3/d/zMPPpd8LsrVzJH/m5KuO2HZrZI2u/tSSZuzxwAmkYbhd/enJR04YfEqSRuy+xsk3VBwXwBK1upn/vnuPihJ2e25xbUEoAqln9tvZr2SeiVpht5Q9u4ANKnVI/8+M1sgSdntUN6K7r7O3bvdvbsrMWkjgGq1Gv6Nknqy+z2SniimHQBVaRh+M3tAUp+kPzazPWb2EUl3SLrWzHZJujZ7DGASafiZ391X55SuKbgXlGD/Wz1ZbzSO30jPjz+arC97nLH8TsUZfkBQhB8IivADQRF+ICjCDwRF+IGguHT3aeDIpvNza30X3dng2emhvsv6epL1i9f8d7LO5bc7F0d+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcf5J4Iwli5P1z73l33Nrsxv8ZHfb4fS+z/9ceqR+dJgrtk9WHPmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjG+SeBCx/6VbJ++bTW/4av3vz3yfqy/9ra8rbR2TjyA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQDcf5zWy9pPdJGnL3S7Nlt0n6O0mvZqutdfcny2rydDfcc0Wy/tn5ja69Pz230jPw58lnXvyp3ck6190/fTVz5P+mpOsmWH63u6/I/hF8YJJpGH53f1rSgQp6AVChdj7z32JmL5rZejObXVhHACrRavi/JulCSSskDUrK/VBqZr1m1m9m/UfV4IJxACrTUvjdfZ+7j7r7MUlfl7Qyse46d+929+6uxBdTAKrVUvjNbMG4hx+QtKOYdgBUpZmhvgckXSVprpntkfQZSVeZ2QpJLmlA0sdK7BFACRqG391XT7D4nhJ6OW2dsfCPkvV3/9OWZP2sKa1/XOp76S3J+rJhfq8fFWf4AUERfiAowg8ERfiBoAg/EBThB4Li0t0V2Ll2UbL++Ju+3db2r95+Y26Nn+wiD0d+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcf4KbHv/3Q3WaO8KR2f/w7Hc2sjwcFvbxumLIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4/2ng6Pyzc2tdRxZW2MnJRl/dn1vzw+np22x6+vyHqfPmttSTJI3OOydZ37VmWsvbboaPWm7ton9scA2GgwcL6YEjPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E1XCc38wWSbpX0pskHZO0zt2/YmZzJH1L0mJJA5Jucnd+PF6D7zy8vu4Wcr3zPyea4X3M/n1vTD539rxDyfqWt93fUk+dbvk/35KsL/lUXyH7aebIPyJpjbtfLOkdkj5uZssl3Spps7svlbQ5ewxgkmgYfncfdPfns/uHJO2UtFDSKkkbstU2SLqhrCYBFO+UPvOb2WJJl0vaImm+uw9KY38gJJ1bdHMAytN0+M3sLEmPSPqkuzd9crGZ9ZpZv5n1H1X6XG4A1Wkq/GbWpbHg3+fuj2aL95nZgqy+QNLQRM9193Xu3u3u3V1tXqgSQHEaht/MTNI9kna6+13jShsl9WT3eyQ9UXx7AMpi7p5ewexdkp6RtF1jQ32StFZjn/sfkvRmSa9IutHdD6S29Uab42+3a9rtedL5v+9fkKxvvvThijqJ5Xd+JLd21PMvd96M61+8OVn/zQut/9x4wbMjyfr0727NrW3xzTroB/J/LzxOw3F+d39WUt7G4iUZOE1whh8QFOEHgiL8QFCEHwiK8ANBEX4gKC7dXYEz/+J/kvVLvpD+CaeX+F9p1kXJUzNK/dnsJc98OFn3V2a2tf0lD7+WX3xue1vbnq1dbdU7AUd+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiq4e/5ixT19/xAVU7l9/wc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiCohuE3s0Vm9iMz22lmPzWzT2TLbzOzX5nZC9m/68tvF0BRmpkOYkTSGnd/3sxmSdpmZpuy2t3u/qXy2gNQlobhd/dBSYPZ/UNmtlPSwrIbA1CuU/rMb2aLJV0uaUu26BYze9HM1pvZ7Jzn9JpZv5n1H9XhtpoFUJymw29mZ0l6RNIn3f2gpK9JulDSCo29M7hzoue5+zp373b37i5NL6BlAEVoKvxm1qWx4N/n7o9Kkrvvc/dRdz8m6euSVpbXJoCiNfNtv0m6R9JOd79r3PIF41b7gKQdxbcHoCzNfNt/paQPSdpuZi9ky9ZKWm1mKyS5pAFJHyulQwClaObb/mclTXQd8CeLbwdAVTjDDwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EJS5e3U7M3tV0v+OWzRX0v7KGjg1ndpbp/Yl0VuriuztfHef18yKlYb/pJ2b9bt7d20NJHRqb53al0RvraqrN972A0ERfiCousO/rub9p3Rqb53al0Rvraqlt1o/8wOoT91HfgA1qSX8Znadmf3czHab2a119JDHzAbMbHs283B/zb2sN7MhM9sxbtkcM9tkZruy2wmnSaupt46YuTkxs3Str12nzXhd+dt+M5sq6ReSrpW0R9JWSavd/aVKG8lhZgOSut299jFhM/tTSa9JutfdL82WfVHSAXe/I/vDOdvdP90hvd0m6bW6Z27OJpRZMH5maUk3SLpZNb52ib5uUg2vWx1H/pWSdrv7y+5+RNKDklbV0EfHc/enJR04YfEqSRuy+xs09j9P5XJ66wjuPujuz2f3D0k6PrN0ra9doq9a1BH+hZJ+Oe7xHnXWlN8u6Skz22ZmvXU3M4H52bTpx6dPP7fmfk7UcObmKp0ws3THvHatzHhdtDrCP9HsP5005HClu79V0nslfTx7e4vmNDVzc1UmmFm6I7Q643XR6gj/HkmLxj0+T9LeGvqYkLvvzW6HJD2mzpt9eN/xSVKz26Ga+3ldJ83cPNHM0uqA166TZryuI/xbJS01swvMbJqkD0raWEMfJzGzmdkXMTKzmZLeo86bfXijpJ7sfo+kJ2rs5Q90yszNeTNLq+bXrtNmvK7lJJ9sKOPLkqZKWu/ut1fexATMbInGjvbS2CSm99fZm5k9IOkqjf3qa5+kz0h6XNJDkt4s6RVJN7p75V+85fR2lcbeur4+c/Pxz9gV9/YuSc9I2i7pWLZ4rcY+X9f22iX6Wq0aXjfO8AOC4gw/ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANB/R5UEeYO44sn+QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADH9JREFUeJzt3X/sXXV9x/Hne6W0EySCWlZLB+gIWQVX59dqUuNwDAILG/iHxP5husRYkskyE/+Q8I/EZYYsijP74VJGY90EdUNsk+EmIVvQjDAKI/ywMgjpsLRpJaAgk0Lb9/74npov8L3n++29595zv7yfj+Sbe+75nHPPO7d9fT/n3M/53k9kJpLq+ZW+C5DUD8MvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqmoEyZ5sBNjRa7kpEkeUirlRV7gpTwUi9l2pPBHxCXAl4FlwN9n5vVt26/kJN4XF45ySEkt7sk7F73t0Kf9EbEM+BvgUmAdsCki1g37epIma5Rr/g3A45n5RGa+BHwDuLybsiSN2yjhXwP8eM7zvc26V4iILRGxKyJ2vcyhEQ4nqUujhH++DxVe8/fBmbk1M2cyc2Y5K0Y4nKQujRL+vcDaOc/PAPaNVo6kSRkl/PcC50TE2RFxIvBRYGc3ZUkat6GH+jLzcERcDfwbs0N92zLzkc4qkzRWI43zZ+btwO0d1SJpgry9VyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiJjpFt+qJ97xzYNu/7PyH1n3P/7urW9vX/tl/DlWTZtnzS0UZfqkowy8VZfilogy/VJThl4oy/FJRI43zR8Qe4HngCHA4M2e6KEqvHwffe8rAtsMcad33Dfuy63I0Rxc3+XwoM5/u4HUkTZCn/VJRo4Y/ge9FxH0RsaWLgiRNxqin/Rszc19ErALuiIgfZeZdczdofilsAVjJG0Y8nKSujNTzZ+a+5vEgcBuwYZ5ttmbmTGbOLGfFKIeT1KGhwx8RJ0XEG48tAxcDD3dVmKTxGuW0/3Tgtog49jo3Z+a/dlKVpLEbOvyZ+QTwWx3WotehZ981eCx/7+FDrfu++aa7uy5HczjUJxVl+KWiDL9UlOGXijL8UlGGXyrKr+7WSHLj+tb27192w8C237nrT1r3/Q3+e6iatDj2/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9UlOP8Gskz6361tX31ssFf3bbmn5d3XY6Ogz2/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxXlOL9GcuEft3+99ndeeNPAtpP/49HWfdsn8Nao7Pmlogy/VJThl4oy/FJRhl8qyvBLRRl+qagFx/kjYhtwGXAwM89r1p0GfBM4C9gDXJmZz46vTPVl2TvPbW3//KpbWttveu6MgW1HfvqzoWpSNxbT838VuORV664B7szMc4A7m+eSlpAFw5+ZdwHPvGr15cD2Znk7cEXHdUkas2Gv+U/PzP0AzeOq7kqSNAljv7c/IrYAWwBWMvj73CRN1rA9/4GIWA3QPB4ctGFmbs3MmcycWc6KIQ8nqWvDhn8nsLlZ3gzs6KYcSZOyYPgj4hbgbuDciNgbER8HrgcuiojHgIua55KWkAWv+TNz04CmCzuuRVPoqYvePNL+9z1/ZkvrL0Z6bY3GO/ykogy/VJThl4oy/FJRhl8qyvBLRfnV3Wr13LqXR9r/gb9eP7DtTbR/7bfGy55fKsrwS0UZfqkowy8VZfilogy/VJThl4pynL+4Q5e+t7V9x8V/1dr+uaff09p+2q0PDmw72rqnxs2eXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKcpy/uL2/2/5f4F0nrmxt37zn/Nb2VS/86Lhr0mTY80tFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUQuO80fENuAy4GBmntesuw74BPCTZrNrM/P2cRWp8XnreQdb249k+1/dn7Dj1C7L0QQtpuf/KnDJPOu/lJnrmx+DLy0xC4Y/M+8CnplALZImaJRr/qsj4sGI2BYRnvtJS8yw4f8K8A5gPbAf+OKgDSNiS0TsiohdL3NoyMNJ6tpQ4c/MA5l5JDOPAjcCG1q23ZqZM5k5s5wVw9YpqWNDhT8iVs95+mHg4W7KkTQpixnquwW4AHhLROwFPgtcEBHrgQT2AFeNsUZJY7Bg+DNz0zyrbxpDLRqDE84+s7X9C+f+U2v7jT9b29p+2ra7j7smTQfv8JOKMvxSUYZfKsrwS0UZfqkowy8V5Vd3v849dtXbWtvfv8BNl5+4/0Ot7Wu9v2vJsueXijL8UlGGXyrK8EtFGX6pKMMvFWX4paIc53+dO7r2xZH2/8VP26fo1tJlzy8VZfilogy/VJThl4oy/FJRhl8qyvBLRTnO/zr3t+/7x5H2X/PdZR1Vomljzy8VZfilogy/VJThl4oy/FJRhl8qyvBLRS04zh8Ra4GvAb8GHAW2ZuaXI+I04JvAWcAe4MrMfHZ8pWqQF/9gw8C2D6z8rwX29laPqhbT8x8GPp2Zvwm8H/hkRKwDrgHuzMxzgDub55KWiAXDn5n7M/P+Zvl5YDewBrgc2N5sth24YlxFSurecV3zR8RZwLuBe4DTM3M/zP6CAFZ1XZyk8Vl0+CPiZOBW4FOZ+dxx7LclInZFxK6XOTRMjZLGYFHhj4jlzAb/65n57Wb1gYhY3bSvBg7Ot29mbs3MmcycWc4Cs0JKmpgFwx8RAdwE7M7MG+Y07QQ2N8ubgR3dlydpXBYzzrMR+BjwUEQ80Ky7Frge+FZEfBx4EvjIeErUQp78wxzYtiLa/4k/9/T5re0n77ivtX3wkTXtFgx/Zv4AiAHNF3ZbjqRJ8Q4/qSjDLxVl+KWiDL9UlOGXijL8UlH+PecSsOyUU1rbP7Px9qFf++bvfrC1/e2H7x76tTXd7Pmlogy/VJThl4oy/FJRhl8qyvBLRRl+qSjH+ZeAo4fav/7sh//3toFtv/fUTOu+53z+kdb2I62tWsrs+aWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMf5l4BcYJz/0Zah/BP539Z9Hcevy55fKsrwS0UZfqkowy8VZfilogy/VJThl4paMPwRsTYi/j0idkfEIxHxp8366yLiqYh4oPn5/fGXK6kri7nJ5zDw6cy8PyLeCNwXEXc0bV/KzC+MrzxJ47Jg+DNzP7C/WX4+InYDa8ZdmKTxOq5r/og4C3g3cE+z6uqIeDAitkXEqQP22RIRuyJi18u036YqaXIWHf6IOBm4FfhUZj4HfAV4B7Ce2TODL863X2ZuzcyZzJxZzooOSpbUhUWFPyKWMxv8r2fmtwEy80BmHsnMo8CNwIbxlSmpa4v5tD+Am4DdmXnDnPWr52z2YeDh7suTNC6L+bR/I/Ax4KGIeKBZdy2wKSLWAwnsAa4aS4WSxmIxn/b/AIh5moafFF5S77zDTyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilogy/VFRk5uQOFvETeMWc0W8Bnp5YAcdnWmub1rrA2obVZW1nZuZbF7PhRMP/moNH7MrMltnl+zOttU1rXWBtw+qrNk/7paIMv1RU3+Hf2vPx20xrbdNaF1jbsHqprddrfkn96bvnl9STXsIfEZdExKMR8XhEXNNHDYNExJ6IeKiZeXhXz7Vsi4iDEfHwnHWnRcQdEfFY8zjvNGk91TYVMze3zCzd63s3bTNeT/y0PyKWAf8DXATsBe4FNmXmDydayAARsQeYyczex4Qj4oPAz4GvZeZ5zbq/AJ7JzOubX5ynZuZnpqS264Cf9z1zczOhzOq5M0sDVwB/RI/vXUtdV9LD+9ZHz78BeDwzn8jMl4BvAJf3UMfUy8y7gGdetfpyYHuzvJ3Z/zwTN6C2qZCZ+zPz/mb5eeDYzNK9vnctdfWij/CvAX485/lepmvK7wS+FxH3RcSWvouZx+nNtOnHpk9f1XM9r7bgzM2T9KqZpafmvRtmxuuu9RH++Wb/maYhh42Z+dvApcAnm9NbLc6iZm6elHlmlp4Kw8543bU+wr8XWDvn+RnAvh7qmFdm7mseDwK3MX2zDx84Nklq83iw53p+aZpmbp5vZmmm4L2bphmv+wj/vcA5EXF2RJwIfBTY2UMdrxERJzUfxBARJwEXM32zD+8ENjfLm4EdPdbyCtMyc/OgmaXp+b2bthmve7nJpxnK+EtgGbAtM/984kXMIyLezmxvD7OTmN7cZ20RcQtwAbN/9XUA+CzwHeBbwK8DTwIfycyJf/A2oLYLmD11/eXMzceusSdc2weA7wMPAUeb1dcye33d23vXUtcmenjfvMNPKso7/KSiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMMvFfX/7rNytWwRJ6EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADlJJREFUeJzt3X+MHPV5x/HPk/P5DkyaQsFwsd0aIkNDLOGEk4OgTU0RFlS0Nk1wcdvISWmPJFA1UqqUWKRAlTQWbQhJk5BcgmWnIsRpDbEj0TToGuREqTCHS21j88OiDhx2bZBRbaJwPvue/nFjcpib7+7tzuzs3fN+SdbtzjOz89zC52Z3vzvzNXcXgHjeUnUDAKpB+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBDWjlTubaV3erVmt3CUQymv6uY76sNWzblPhN7OrJH1RUoekb7r7mtT63Zql99oVzewSQMKjPlD3ug2/7DezDklfkXS1pAslrTSzCxt9PACt1cx7/sWS9rj7c+5+VNJ3JC0rpi0AZWsm/HMkvTDu/lC27A3MrM/MBs1scETDTewOQJGaCf9EHyq86fxgd+9391537+1UVxO7A1CkZsI/JGneuPtzJe1rrh0ArdJM+B+TtMDMzjWzmZKul7S5mLYAlK3hoT53P2ZmN0v6d40N9a119ycL6wxAqZoa53f3hyQ9VFAvAFqIr/cCQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFRLp+hG63X86tuS9ae/fF6y/tTl30zWbz14cbK+40/Oz60d3/VMcluUiyM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTV1Di/me2VdETScUnH3L23iKZQnNFz5ybrO5Z8PVkf8fTjf2b248n6Rddemlubxzh/pYr4ks/l7v5yAY8DoIV42Q8E1Wz4XdIPzexxM+sroiEArdHsy/7L3H2fmc2W9LCZPeXuW8avkP1R6JOkbp3a5O4AFKWpI7+778t+HpT0oKTFE6zT7+697t7bqa5mdgegQA2H38xmmdlbT9yWtFTSzqIaA1CuZl72ny3pQTM78TjfdvcfFNIVgNI1HH53f07SRQX2ggbNmJc/ln9u/54WdoKphKE+ICjCDwRF+IGgCD8QFOEHgiL8QFBcunsKeP5v80+LlaSLr9qVW7uz58dFtzMpp136Um7thU+nf68ztx9L1k/ZtLWhnjCGIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4/xSw/cZ/StZH/HiLOpm8Ry66L79Y44TwB3/ek6yvPbI8WZ/xH+nLikfHkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmKcvw10PpIez+60jhZ1Mnn/dXQ0Wd87clZu7dpZh5LbrjjtYLr+z/3J+jVzLk7Wo+PIDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANB1RznN7O1kq6RdNDdF2bLzpC0QdJ8SXslrXD3V8prc2r7xfLFyfqHe/4lWa91vn6Z5/MvHPhIsn7WQFey3vV/+b19akn62LPjui8l67UMfSp/XoC5n/tpU489HdRz5F8n6aqTlt0iacDdF0gayO4DmEJqht/dt0g6+atYyyStz26vl5S+pAqAttPoe/6z3X2/JGU/ZxfXEoBWKP27/WbWJ6lPkrp1atm7A1CnRo/8B8ysR5Kyn7lnYLh7v7v3untvp9IfDgFonUbDv1nSquz2KkmbimkHQKvUDL+Z3S/pPyVdYGZDZnaDpDWSrjSzZyVdmd0HMIXUfM/v7itzSlcU3MuU1fGuC5L1z9yVPu+8d+bRWnuYZEe/VOva97f+6P3J+js/+VSyfvzw4Un3dMIFz56frG/9g+5kfXHXa8n6v330ztza0u5PJred//fpa/778HCyPhXwDT8gKMIPBEX4gaAIPxAU4QeCIvxAUFy6uwCjM9NPY+2hvOb82c9OPunyl4780SnJbc8f2pqslzn59/FdzyTrH1uXPp148Ma7k/WejvzffdsN6W3f/8CqZN3/e3eyPhVw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnnwJWH+hN1g//+a/l1o4PPVt0Oy0zf+PLyfqnl1+SrK8557Ei25l2OPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCM87dApzV+6W1J2v4er7HG1B3LTzJLlme8ZTRZb+Z533dHun7ONJialiM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVc5zfzNZKukbSQXdfmC27XdJfSHopW221uz9UVpPt7umPnpqsj3iZV7+fvvb+Yf51CiTpX89Kzzkw4vnj/LX+m7z9tmRZ6W8YTA31HPnXSZpoVogvuPui7F/Y4ANTVc3wu/sWSYda0AuAFmrmPf/NZrbdzNaa2emFdQSgJRoN/z2S3iFpkaT9kj6ft6KZ9ZnZoJkNjmi4wd0BKFpD4Xf3A+5+3N1HJX1D0uLEuv3u3uvuvZ3qarRPAAVrKPxm1jPu7rWSdhbTDoBWqWeo735JSySdaWZDkm6TtMTMFklySXsl3VhijwBKUDP87r5ygsX3ltDLlHXrb3+/6hba1ox5c3NrRy5+e3Lbr334q0W387qtw93Juh09Vtq+2wXf8AOCIvxAUIQfCIrwA0ERfiAowg8ExaW7Uapdd5yTW3ty6ZdL3ffGV8/Mrd3z19clt+3enT5deDrgyA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOj6Z0PtKTrH+uZ2OLOnmzdS9emlvr/v70H8evhSM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTFOH8BOiw9YXOn5U8VXY/Df3xJw9ve8Xfpq6xffsprDT+2VPt3S0+F3dzzUov/7oulPv5Ux5EfCIrwA0ERfiAowg8ERfiBoAg/EBThB4KqOc5vZvMkfUvSOZJGJfW7+xfN7AxJGyTNl7RX0gp3f6W8VtvXmg0fSNZX3HB3U4+/5R++kqynx9LTRrzhTet8/MZ7q2XhwEeS9QXaVtq+p4N6jvzHJH3C3d8p6RJJN5nZhZJukTTg7gskDWT3AUwRNcPv7vvdfVt2+4ik3ZLmSFomaX222npJy8tqEkDxJvWe38zmS3q3pEclne3u+6WxPxCSZhfdHIDy1B1+MztN0kZJH3f3w5PYrs/MBs1scETDjfQIoAR1hd/MOjUW/Pvc/YFs8QEz68nqPZIOTrStu/e7e6+793aqq4ieARSgZvjNzCTdK2m3u981rrRZ0qrs9ipJm4pvD0BZ6jml9zJJH5S0w8yeyJatlrRG0nfN7AZJz0tKz3k8jZ234eVkfeufdifri7uaO622nW0dzv/d+//3d5LbvvKx/Om9Jek3/2dPsl7eIOP0UDP87v4TSZZTvqLYdgC0Ct/wA4Ii/EBQhB8IivADQRF+ICjCDwRl7iWf0znOr9gZ/l6LNzr4i2WLk/UXfj996e9nrv56sl7mabO11Lp090Vf/cvc2rzP/rTodsJ71Ad02A/lDc2/AUd+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKKbpb4JRNW5P182tcBuV9K29K1js/dCC39oN3bUhuu3Tn9cn66Lr0pRm9xojy/Cdeyq1xvn21OPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCczw9MI5zPD6Amwg8ERfiBoAg/EBThB4Ii/EBQhB8Iqmb4zWyemf3IzHab2ZNm9lfZ8tvN7EUzeyL793vltwugKPVczOOYpE+4+zYze6ukx83s4az2BXf/x/LaA1CWmuF39/2S9me3j5jZbklzym4MQLkm9Z7fzOZLerekR7NFN5vZdjNba2an52zTZ2aDZjY4ouGmmgVQnLrDb2anSdoo6ePufljSPZLeIWmRxl4ZfH6i7dy939173b23U10FtAygCHWF38w6NRb8+9z9AUly9wPuftzdRyV9Q1J6NkoAbaWeT/tN0r2Sdrv7XeOW94xb7VpJO4tvD0BZ6vm0/zJJH5S0w8yeyJatlrTSzBZJckl7Jd1YSocASlHPp/0/kTTR+cEPFd8OgFbhG35AUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgWjpFt5m9JOln4xadKenlljUwOe3aW7v2JdFbo4rs7Tfc/ax6Vmxp+N+0c7NBd++trIGEdu2tXfuS6K1RVfXGy34gKMIPBFV1+Psr3n9Ku/bWrn1J9NaoSnqr9D0/gOpUfeQHUJFKwm9mV5nZ02a2x8xuqaKHPGa218x2ZDMPD1bcy1ozO2hmO8ctO8PMHjazZ7OfE06TVlFvbTFzc2Jm6Uqfu3ab8brlL/vNrEPSM5KulDQk6TFJK919V0sbyWFmeyX1unvlY8Jm9j5Jr0r6lrsvzJbdKemQu6/J/nCe7u5/0ya93S7p1apnbs4mlOkZP7O0pOWSPqQKn7tEXytUwfNWxZF/saQ97v6cux+V9B1Jyyroo+25+xZJh05avEzS+uz2eo39z9NyOb21BXff7+7bsttHJJ2YWbrS5y7RVyWqCP8cSS+Muz+k9pry2yX90MweN7O+qpuZwNnZtOknpk+fXXE/J6s5c3MrnTSzdNs8d43MeF20KsI/0ew/7TTkcJm7v0fS1ZJuyl7eoj51zdzcKhPMLN0WGp3xumhVhH9I0rxx9+dK2ldBHxNy933Zz4OSHlT7zT584MQkqdnPgxX387p2mrl5opml1QbPXTvNeF1F+B+TtMDMzjWzmZKul7S5gj7exMxmZR/EyMxmSVqq9pt9eLOkVdntVZI2VdjLG7TLzM15M0ur4ueu3Wa8ruRLPtlQxt2SOiStdffPtryJCZjZeRo72ktjk5h+u8rezOx+SUs0dtbXAUm3SfqepO9K+nVJz0u6zt1b/sFbTm9LNPbS9fWZm0+8x25xb78l6ceSdkgazRav1tj768qeu0RfK1XB88Y3/ICg+IYfEBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg/h9YF/1+epKp7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADdhJREFUeJzt3X+MHPV5x/HPw/lsC+MGO/xybIMT6gQc0l7QyoBcVQ4uhCSoNn+ExJWoKyEuUuOmUSKl1IoUK1Ur+iOktKI0R3BsxK9QAcFqUAKxQp1fdTlTKyYxIYhciPHFZ2oDJqX+cff0j5tDZ3Pz3fXuzM7uPe+XhHZ3npmdRyM+ntn7zu7X3F0A4jmt6gYAVIPwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IakY7dzbTZvlszWnnLoFQ/k+/0VE/Yo2s21L4zewaSbdJ6pH0VXe/JbX+bM3RZbaqlV0CSNjh2xpet+nLfjPrkXS7pA9JWiZprZkta/b9ALRXK5/5l0t63t1fcPejkh6QtLqYtgCUrZXwL5T0q0mv92bLTmBm/WY2aGaDx3Skhd0BKFIr4Z/qjwpv+X6wuw+4e83da72a1cLuABSplfDvlbR40utFkva11g6Admkl/E9JWmpm7zSzmZI+LmlrMW0BKFvTQ33uftzM1kv6tsaH+ja5+08K6wxAqVoa53f3xyQ9VlAvANqI23uBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IqqVZes1sSNJhSaOSjrt7rYimcKLRD1yarK8feDC3dsfS3y66nY5x+GOXJ+tn7no5tzb6s+eLbqfrtBT+zAfcPf8oA+hIXPYDQbUafpf0uJntNLP+IhoC0B6tXvavcPd9ZnaOpCfM7Fl33z55hewfhX5Jmq3TW9wdgKK0dOZ3933Z44ikRyQtn2KdAXevuXutV7Na2R2AAjUdfjObY2ZzJ55LulrSM0U1BqBcrVz2nyvpETObeJ/73P1bhXQFoHRNh9/dX5D0uwX2ghy//GD649L8ntfb1Eln+fVHjibrx27Iv7Cdf23R3XQfhvqAoAg/EBThB4Ii/EBQhB8IivADQRXxrT60yHpnJutXXrmrTZ10l7n/PTtZv/7G/8itfffMRcltR195tameuglnfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IinH+DnD4uvRPc//Twn9O1i/+xvrc2lLtaKqnbnBknifrn5r3bG7tybkXp9+ccX4A0xXhB4Ii/EBQhB8IivADQRF+ICjCDwTFOH8b+Iq+ZP32v70tWb/ntQuS9Ys+/1xubTS5ZXe74mrmiGkFZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKruOL+ZbZJ0raQRd78kWzZf0tclLZE0JOl6dz9UXpvd7dBf/m+yvmjG8WT9M3/2kWS999DOU+6pG8xYcF6y/rXzv5WsH3PObSmNHJ3Nkq45adnNkra5+1JJ27LXALpI3fC7+3ZJB09avFrSluz5FklrCu4LQMmavS46192HJSl7PKe4lgC0Q+n39ptZv6R+SZqt08veHYAGNXvm329mCyQpexzJW9HdB9y95u61Xs1qcncAitZs+LdKWpc9Xyfp0WLaAdAudcNvZvdL+pGk95jZXjO7UdItkq4ys59Luip7DaCL1P3M7+5rc0qrCu6la/3PTVck6//2vr9P1u9+9XeS9d7vTM9x/Hp++sXFyfoxT/9awbqhP8itjY4caKqn6YS7IICgCD8QFOEHgiL8QFCEHwiK8ANB8dPdBThtzcvJ+jtmpO9svOu+k780eaJF+uEp99QNet77nmT9nlVfSdaP+LFk/cVb351bm3Nk+k5d3ijO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOP8Deo5++zc2uff/c2W3nvR30zPcfx6nv3TM5P12qz0V3ZvP7QsWZ/zEGP5KZz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvkbZKfPzq198PRXk9suf+qPk/XztKepnrrdWUtOnv/11Nz7i1r6/fVcS+8/3XHmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg6o7zm9kmSddKGnH3S7JlGyXdJGlinuMN7v5YWU12grGDr+TW/urApclt/+jCwWR9+4ILk/Xjw79O1jvZjAvyp9n+Qd8DdbZOn5ve+M+z6mzPOH9KI2f+zZKmmlXiy+7el/03rYMPTEd1w+/u2yW1disWgI7Tymf+9Wb2YzPbZGbzCusIQFs0G/47JF0oqU/SsKQv5a1oZv1mNmhmg8d0pMndAShaU+F39/3uPuruY5LulLQ8se6Au9fcvdar9ISVANqnqfCb2YJJL6+T9Ewx7QBol0aG+u6XtFLSWWa2V9IXJK00sz5JLmlI0idK7BFACeqG393XTrH4rhJ66Whjhw/n1h5/6aLktt/ruy9ZH/73t6W3/8oVyXqZXlnmyfoZS9K/ZXD5O4Zya2Maa6alN1m6NdTBHX5AUIQfCIrwA0ERfiAowg8ERfiBoMy9feMlv2Xz/TJb1bb9tc3y9yXLr258I1l/5JLNyfr8nurujBw80pOsj9Y5f9RmHs2t9Zg11dOENRddmaynhmenqx2+Ta/5wYYOLGd+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKKbqL8F+7k+W3fTi9+Q0rP5Wsv7K0unH+t9/5o5a2f+nh9+bWdl62uaX3jjiOXyTO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOP8HaDnyaeT9bc/2Z4+yvDG0Nz84mWtvbev6EvW7Qe7WtvBNMeZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqjvOb2aLJd0t6TxJY5IG3P02M5sv6euSlkgaknS9ux8qr1V0pcQvyJ/W4rmHcfzWNHL0j0v6rLtfLOlySZ80s2WSbpa0zd2XStqWvQbQJeqG392H3f3p7PlhSXskLZS0WtKWbLUtktaU1SSA4p3SdZeZLZH0fkk7JJ3r7sPS+D8Qks4pujkA5Wk4/GZ2hqSHJH3a3V87he36zWzQzAaP6UgzPQIoQUPhN7NejQf/Xnd/OFu838wWZPUFkkam2tbdB9y95u61XlX3Q5QATlQ3/GZmku6StMfdb51U2ippXfZ8naRHi28PQFka+UrvCkk3SNptZhNjKxsk3SLpQTO7UdKLkj5aTovoaokZ4Mc01r4+8BZ1w+/u31f+aO2qYtsB0C7c4QcERfiBoAg/EBThB4Ii/EBQhB8Iip/uRqnGZjc/ln9glNvBy8SZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpwfpbrnmn/Nre05mr4HYO3mzyXr5+uHTfWEcZz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvlRqi/+4g9za7/5l4XJbc9/iHH8MnHmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg6o7zm9liSXdLOk/SmKQBd7/NzDZKuknSgWzVDe7+WFmNokut2ptbmqP8GsrXyE0+xyV91t2fNrO5knaa2RNZ7cvu/g/ltQegLHXD7+7Dkoaz54fNbI+k9K1ZADreKX3mN7Mlkt4vaUe2aL2Z/djMNpnZvJxt+s1s0MwGj4npl4BO0XD4zewMSQ9J+rS7vybpDkkXSurT+JXBl6bazt0H3L3m7rVezSqgZQBFaCj8Ztar8eDf6+4PS5K773f3UXcfk3SnpOXltQmgaHXDb2Ym6S5Je9z91knLF0xa7TpJzxTfHoCyNPLX/hWSbpC028x2Zcs2SFprZn2SXNKQpE+U0iGAUjTy1/7vS7IpSozpA12MO/yAoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBmbu3b2dmByT9ctKisyS93LYGTk2n9tapfUn01qwie7vA3c9uZMW2hv8tOzcbdPdaZQ0kdGpvndqXRG/Nqqo3LvuBoAg/EFTV4R+oeP8pndpbp/Yl0VuzKumt0s/8AKpT9ZkfQEUqCb+ZXWNmPzOz583s5ip6yGNmQ2a228x2mdlgxb1sMrMRM3tm0rL5ZvaEmf08e5xymrSKettoZi9lx26XmX24ot4Wm9l3zWyPmf3EzP48W17psUv0Vclxa/tlv5n1SHpO0lWS9kp6StJad/9pWxvJYWZDkmruXvmYsJn9vqTXJd3t7pdky/5O0kF3vyX7h3Oeu/9Fh/S2UdLrVc/cnE0os2DyzNKS1kj6E1V47BJ9Xa8KjlsVZ/7lkp539xfc/aikByStrqCPjufu2yUdPGnxaklbsudbNP4/T9vl9NYR3H3Y3Z/Onh+WNDGzdKXHLtFXJaoI/0JJv5r0eq86a8pvl/S4me00s/6qm5nCudm06RPTp59TcT8nqztzczudNLN0xxy7Zma8LloV4Z9q9p9OGnJY4e6XSvqQpE9ml7doTEMzN7fLFDNLd4RmZ7wuWhXh3ytp8aTXiyTtq6CPKbn7vuxxRNIj6rzZh/dPTJKaPY5U3M+bOmnm5qlmllYHHLtOmvG6ivA/JWmpmb3TzGZK+rikrRX08RZmNif7Q4zMbI6kq9V5sw9vlbQue75O0qMV9nKCTpm5OW9maVV87DptxutKbvLJhjL+UVKPpE3u/tdtb2IKZvYujZ/tpfFJTO+rsjczu1/SSo1/62u/pC9I+oakByWdL+lFSR9197b/4S2nt5Uav3R9c+bmic/Ybe7t9yR9T9JuSWPZ4g0a/3xd2bFL9LVWFRw37vADguIOPyAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQf0/UNTPWUuggRYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADMdJREFUeJzt3X/MnXV5x/H3RX0o4YcJBFtrKcJY5yD8AeYBN3ELjoDgWIpmMPsH6xJjTSaZJCYb4R/5Y8twmToWDUmRxpIJaKKMZmGbrHEykq3jKav8WCcw7aS2aWHoKCr9ee2P59Q8lufc5+H8us/T6/1KmnPOfd0/rpz089znnO99zjcyE0n1nNR2A5LaYfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxX1lnEe7ORYmqdw2jgPKZXyOj/hYB6Ihaw7UPgj4lrgLmAJ8KXMvLNp/VM4jffEVYMcUlKDrbllwev2/bI/IpYAXwSuAy4C1kbERf3uT9J4DfKe/3Lghcz8XmYeBB4E1gynLUmjNkj4VwIvznm8q7PsF0TE+oiYiYiZQxwY4HCShmmQ8M/3ocIbvh+cmRsyczozp6dYOsDhJA3TIOHfBaya8/gcYPdg7Ugal0HC/wSwOiLOj4iTgY8Am4fTlqRR63uoLzMPR8QtwD8yO9S3MTOfHVpnkkZqoHH+zHwEeGRIvUgaIy/vlYoy/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qaiBZumNiJ3AfuAIcDgzp4fRlE4cP/79X+9a23rn3Y3bXvTFP2ysn/uZf2+s5+HDjfXqBgp/x/sz8+Uh7EfSGPmyXypq0PAn8M2I2BYR64fRkKTxGPRl/xWZuTsilgGPRsR/ZeZjc1fo/FFYD3AKpw54OEnDMtCZPzN3d273AQ8Bl8+zzobMnM7M6SmWDnI4SUPUd/gj4rSIOOPYfeAa4JlhNSZptAZ52b8ceCgiju3n/sz8h6F0JWnkIjPHdrC3xln5nrhqbMfT6L1l5Tsa65/89qNda9ecemigY1/3rt9orB/dv3+g/S9GW3MLr+YrsZB1HeqTijL8UlGGXyrK8EtFGX6pKMMvFTWMb/WpsH0feGdjfZDhvHfP/F5j/W2vPdf3vuWZXyrL8EtFGX6pKMMvFWX4paIMv1SU4ZeKcpxfjU46tfmn1z7wR4+P7NhLHzyzeYUxfh39ROSZXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKcpxfjQ6898LG+p8uu7fvff/06MHG+lvv/7e+963ePPNLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlE9x/kjYiNwPbAvMy/uLDsL+CpwHrATuCkzfzS6NtWW7394ycj2/bvP39Bjjd0jO7YWdub/MnDtcctuA7Zk5mpgS+expEWkZ/gz8zHgleMWrwE2de5vAnr9CZc0Yfp9z788M/cAdG6XDa8lSeMw8mv7I2I9sB7gFJp/D07S+PR75t8bESsAOrf7uq2YmRsyczozp6dY2ufhJA1bv+HfDKzr3F8HPDycdiSNS8/wR8QDwL8C74qIXRHxUeBO4OqIeB64uvNY0iLS8z1/Zq7tUrpqyL1oAv32Zd8ZaPv/O/qzrrVDdyxv3PYkx/lHyiv8pKIMv1SU4ZeKMvxSUYZfKsrwS0X5093FHfjgZY31L6y8Z6D97zrcvXbSt/9joH1rMJ75paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkox/mL23vZ1Ej3/zt/d2vX2mq2jvTYauaZXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKcpy/uJMvHWxm9R0Hf9pY/9W/frlr7chAR9agPPNLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlE9x/kjYiNwPbAvMy/uLLsD+BjwUme12zPzkVE1qf69fv3ljfWZy+7usYcljdXvHlrWWD/y3H/32L/aspAz/5eBa+dZ/vnMvKTzz+BLi0zP8GfmY8ArY+hF0hgN8p7/loh4KiI2RsSZQ+tI0lj0G/67gQuAS4A9wGe7rRgR6yNiJiJmDnGgz8NJGra+wp+ZezPzSGYeBe4Bun6qlJkbMnM6M6enWNpvn5KGrK/wR8SKOQ8/BDwznHYkjctChvoeAK4Ezo6IXcCngSsj4hIggZ3Ax0fYo6QR6Bn+zFw7z+J7R9CLRuBnZzeP009Fc72XP9724cb6+Tw10P41Ol7hJxVl+KWiDL9UlOGXijL8UlGGXyrKn+4+wR244ccDbd/rp7nP+dJop/jW6Hjml4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiHOc/ASz5lQu61mYu+5teWzdW//61ixvrU/+0rcf+Nak880tFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUY7znwD2vr/7NNmD/jT3F751dWN9NVsH2r/a45lfKsrwS0UZfqkowy8VZfilogy/VJThl4rqOc4fEauA+4C3A0eBDZl5V0ScBXwVOA/YCdyUmT8aXavq5vWzou9ttx042Fi/8DO7GuuH+z6y2raQM/9h4FOZeSHwa8AnIuIi4DZgS2auBrZ0HktaJHqGPzP3ZOaTnfv7gR3ASmANsKmz2ibghlE1KWn43tR7/og4D7gU2Aosz8w9MPsHAuh+jamkibPg8EfE6cDXgVsz89U3sd36iJiJiJlDHOinR0kjsKDwR8QUs8H/SmZ+o7N4b0Ss6NRXAPvm2zYzN2TmdGZOT7F0GD1LGoKe4Y+IAO4FdmTm5+aUNgPrOvfXAQ8Pvz1Jo7KQr/ReAdwMPB0R2zvLbgfuBL4WER8FfgDcOJoW1cuy3/ph39tufvXSxvqRl17ue9+abD3Dn5mPA90Gkq8abjuSxsUr/KSiDL9UlOGXijL8UlGGXyrK8EtF+dPdi0Asbb4ycs07vtP3vv/34OmN9TzgJdknKs/8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1SU4/yLwZEjjeUNO97XtXbre3c2bvvPL/5yY30lzzbWtXh55peKMvxSUYZfKsrwS0UZfqkowy8VZfilohznXwTycPNE2Ofd9pOutQv//ObGbWP7GX31pMXPM79UlOGXijL8UlGGXyrK8EtFGX6pKMMvFdVznD8iVgH3AW8HjgIbMvOuiLgD+BjwUmfV2zPzkVE1qu6OvPD9rrVzbxxjI1pUFnKRz2HgU5n5ZEScAWyLiEc7tc9n5l+Orj1Jo9Iz/Jm5B9jTub8/InYAK0fdmKTRelPv+SPiPOBSYGtn0S0R8VREbIyIM7tssz4iZiJi5hBO/SRNigWHPyJOB74O3JqZrwJ3AxcAlzD7yuCz822XmRsyczozp6donnNO0vgsKPwRMcVs8L+Smd8AyMy9mXkkM48C9wCXj65NScPWM/wREcC9wI7M/Nyc5SvmrPYh4JnhtydpVBbyaf8VwM3A0xGxvbPsdmBtRFwCJLAT+PhIOpQ0Egv5tP9xIOYpOaYvLWJe4ScVZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyoqMnN8B4t4CfifOYvOBl4eWwNvzqT2Nql9gb31a5i9vTMz37aQFcca/jccPGImM6dba6DBpPY2qX2BvfWrrd582S8VZfilotoO/4aWj99kUnub1L7A3vrVSm+tvueX1J62z/ySWtJK+CPi2oj4bkS8EBG3tdFDNxGxMyKejojtETHTci8bI2JfRDwzZ9lZEfFoRDzfuZ13mrSWersjIn7Yee62R8QHW+ptVUR8KyJ2RMSzEfHJzvJWn7uGvlp53sb+sj8ilgDPAVcDu4AngLWZ+Z9jbaSLiNgJTGdm62PCEfGbwGvAfZl5cWfZXwCvZOadnT+cZ2bmn0xIb3cAr7U9c3NnQpkVc2eWBm4A/oAWn7uGvm6iheetjTP/5cALmfm9zDwIPAisaaGPiZeZjwGvHLd4DbCpc38Ts/95xq5LbxMhM/dk5pOd+/uBYzNLt/rcNfTVijbCvxJ4cc7jXUzWlN8JfDMitkXE+rabmcfyzrTpx6ZPX9ZyP8frOXPzOB03s/TEPHf9zHg9bG2Ef77ZfyZpyOGKzHw3cB3wic7LWy3MgmZuHpd5ZpaeCP3OeD1sbYR/F7BqzuNzgN0t9DGvzNzdud0HPMTkzT6899gkqZ3bfS3383OTNHPzfDNLMwHP3STNeN1G+J8AVkfE+RFxMvARYHMLfbxBRJzW+SCGiDgNuIbJm314M7Cuc38d8HCLvfyCSZm5udvM0rT83E3ajNetXOTTGcr4K2AJsDEz/2zsTcwjIn6J2bM9zE5ien+bvUXEA8CVzH7ray/waeBvga8B5wI/AG7MzLF/8NaltyuZfen685mbj73HHnNv7wP+BXgaONpZfDuz769be+4a+lpLC8+bV/hJRXmFn1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilov4fFsePPEpKY3UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADjlJREFUeJzt3X2MXGUVx/HfYbtsoUXTIn2xFIpQqbXBgpuCVrFIIIBo0QihQawRWEjAaGKiWElojCTEKNIYfFntQjGIxQCliY2CDVoIUllqQykF2zSVLi3dQtEWkb7sHv/YW13LzjPTmTtzZ3u+n4TMzD1z7z0Z+ts7M8+985i7C0A8RxXdAIBiEH4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0GNaOTOjrY2H6lRjdwlEMrb+pf2+V6r5Lk1hd/MLpK0SFKLpF+4++2p54/UKJ1t59eySwAJq31lxc+t+m2/mbVIukvSxZKmS5pnZtOr3R6AxqrlM/8sSZvcfbO775P0a0lz82kLQL3VEv5JkrYOetyTLfs/ZtZhZt1m1r1fe2vYHYA81RL+ob5UeMf1we7e6e7t7t7eqrYadgcgT7WEv0fS5EGPT5S0rbZ2ADRKLeF/RtJUMzvFzI6WdKWk5fm0BaDeqh7qc/cDZnaTpN9rYKivy93X59YZgLqqaZzf3VdIWpFTLwAaiNN7gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKqmWXrNbIukPZL6JB1w9/Y8mopmxMmTk/VxS/+RrP/p2ekla9N+nF63b/1LyfqRquWEE5L11y8+LVkfs3RNsu579x52T41WU/gz57n7azlsB0AD8bYfCKrW8LukR83sWTPryKMhAI1R69v+2e6+zczGSXrMzF5091WDn5D9UeiQpJE6tsbdAchLTUd+d9+W3fZKeljSrCGe0+nu7e7e3qq2WnYHIEdVh9/MRpnZcQfvS7pQ0vN5NQagvmp52z9e0sNmdnA7v3L33+XSFYC6qzr87r5Z0ody7OWINWLC+GT9O398MFk/vbU/Wf/k6xNK1vrWb0yueyRLjeVf9WR6nP6ckQ8n6zeuuz6987+uT9ebAEN9QFCEHwiK8ANBEX4gKMIPBEX4gaDyuKovvBEnTkrW3730rWT9jKNbkvXT/3BDsj51fnrYKqoN351SsnbF6PQpKWfd+Y1k/b1/faqalpoKR34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpx/hy8MTv909vLptxV0/Y/cEtvsn6gpq0PX/6R9BXlmy79WcnaJ9Zdnlx3cteLyXpfsjo8cOQHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAY569QahrtnXPfrmnb7d//SrI+Yevwv3a8GuXG8W+5b0nV237zt6V/7lySRr2+ueptDxcc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqLLj/GbWJelSSb3uPiNbNlbSUklTJG2RdIW7v1G/Nou3ddHokrWNs+5JrntL78xkfdLd6emcj4Rrx6vxypxRyfrstvTU5TOeml+ydtKPYp47MVglR/57JF10yLKbJa1096mSVmaPAQwjZcPv7qsk7Tpk8VxJB0+vWiLpspz7AlBn1X7mH+/u2yUpux2XX0sAGqHu5/abWYekDkkaqWPrvTsAFar2yL/DzCZKUnZb8hcm3b3T3dvdvb1VbVXuDkDeqg3/ckkHv0qdL+mRfNoB0Chlw29m90v6s6TTzazHzK6RdLukC8xso6QLsscAhpGyn/ndfV6J0vk599LU3K1kbb+nR+JXvz4lWW/5d/p3+Yezo447rmTtpdumJ9dd9pk7kvV+tSbrJ12+LlmPjjP8gKAIPxAU4QeCIvxAUIQfCIrwA0Hx090NsGLasmT9mj+el6y/vGdisr5vcfpnqOvp1Y97sn7J2WtL1pa/98dltp4eypu99spkfYw2ltl+bBz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvkrNO5Hx5SsPd45Mrnuecekp/BefNLjyfpRKn05sST135Eea6+nsr2p+t7u3zM+WT9+Qfqfb/qHvcGRHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCMvfGjRG/y8b62Xbk/eL3iAnp8ejdH52SrPdcmP5/sOnTP03Wn95buvaFR29Irlurqfcmdi7pt7/pqnrbZ67+YrI+6XPpqc0jWu0rtdt3pU++yHDkB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgyl7Pb2Zdki6V1OvuM7JlCyVdJ2ln9rQF7r6iXk02uwOv7kjWj30oXX//Q+ntX3LDWYfb0v+2rb9UvW4ljjpjWrqeuN7/u6/NSK578lf/mawfSFZRTiVH/nskXTTE8h+6+8zsv7DBB4arsuF391WSdjWgFwANVMtn/pvM7Dkz6zKzMbl1BKAhqg3/TySdKmmmpO2SflDqiWbWYWbdZta9X+nzwAE0TlXhd/cd7t7n7v2Sfi5pVuK5ne7e7u7trWqrtk8AOasq/GY2eNrYz0p6Pp92ADRKJUN990uaI+k9ZtYj6VZJc8xspiSXtEXS9XXsEUAdlA2/u88bYvHiOvSCYejlW1uS9dTv9j9627nJdUdvfbqqnlAZzvADgiL8QFCEHwiK8ANBEX4gKMIPBMUU3Uh6reMjyfpz59yVrG858O+StWN27quqJ+SDIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4P5LeuuDNmtb//NprS9bGPb6mpm2jNhz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvmR9LMP/zJZ3973VrJ+/J3H5tkOcsSRHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCKjvOb2aTJd0raYKkfkmd7r7IzMZKWippiqQtkq5w9zfq1yrqoedbH03WZ7elr7l/em96HL+Fa/abViVH/gOSvu7uH5B0jqQbzWy6pJslrXT3qZJWZo8BDBNlw+/u2919TXZ/j6QNkiZJmitpSfa0JZIuq1eTAPJ3WJ/5zWyKpDMlrZY03t23SwN/ICSNy7s5APVTcfjNbLSkByV9zd13H8Z6HWbWbWbd+7W3mh4B1EFF4TezVg0E/z53fyhbvMPMJmb1iZJ6h1rX3Tvdvd3d21vVlkfPAHJQNvxmZpIWS9rg7ncMKi2XND+7P1/SI/m3B6BeKrmkd7akqyWtM7O12bIFkm6X9ICZXSPpZUmX16dF1NNV81Ym6/3yZP2a7i8l6ydrXclay/Fjk+tq3PHJct+Gjen1kVQ2/O7+pCQrUT4/33YANApn+AFBEX4gKMIPBEX4gaAIPxAU4QeC4qe7UZP+vvTxo/em0pcMf+raJ5LrLts8MVmf9LlkGWVw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnR002nHt3st5/bunfA/jgqi8n1z1t4b+S9b5kFeVw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnD+733/5Esv7Ct9LX1P959bRkfdqibSVrp776UnLdvrffTtZRG478QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxCUuafnXzezyZLulTRBUr+kTndfZGYLJV0naWf21AXuviK1rXfZWD/bmNUbqJfVvlK7fZdV8txKTvI5IOnr7r7GzI6T9KyZPZbVfuju36+2UQDFKRt+d98uaXt2f4+ZbZA0qd6NAaivw/rMb2ZTJJ0paXW26CYze87MusxsTIl1Osys28y692tvTc0CyE/F4Tez0ZIelPQ1d98t6SeSTpU0UwPvDH4w1Hru3unu7e7e3qq2HFoGkIeKwm9mrRoI/n3u/pAkufsOd+9z935JP5c0q35tAshb2fCbmUlaLGmDu98xaPngy70+K+n5/NsDUC+VfNs/W9LVktaZ2dps2QJJ88xspiSXtEXS9XXpEEBdVPJt/5OShho3TI7pA2hunOEHBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquxPd+e6M7Odkv4+aNF7JL3WsAYOT7P21qx9SfRWrTx7O9ndT6jkiQ0N/zt2btbt7u2FNZDQrL01a18SvVWrqN542w8ERfiBoIoOf2fB+09p1t6atS+J3qpVSG+FfuYHUJyij/wAClJI+M3sIjN7ycw2mdnNRfRQipltMbN1ZrbWzLoL7qXLzHrN7PlBy8aa2WNmtjG7HXKatIJ6W2hmr2Sv3Vozu6Sg3iab2eNmtsHM1pvZV7Plhb52ib4Ked0a/rbfzFok/U3SBZJ6JD0jaZ67v9DQRkowsy2S2t298DFhMztX0puS7nX3Gdmy70na5e63Z384x7j7N5ukt4WS3ix65uZsQpmJg2eWlnSZpC+pwNcu0dcVKuB1K+LIP0vSJnff7O77JP1a0twC+mh67r5K0q5DFs+VtCS7v0QD/3garkRvTcHdt7v7muz+HkkHZ5Yu9LVL9FWIIsI/SdLWQY971FxTfrukR83sWTPrKLqZIYzPpk0/OH36uIL7OVTZmZsb6ZCZpZvmtatmxuu8FRH+oWb/aaYhh9nufpakiyXdmL29RWUqmrm5UYaYWbopVDvjdd6KCH+PpMmDHp8oaVsBfQzJ3bdlt72SHlbzzT684+Akqdltb8H9/Fczzdw81MzSaoLXrplmvC4i/M9Immpmp5jZ0ZKulLS8gD7ewcxGZV/EyMxGSbpQzTf78HJJ87P78yU9UmAv/6dZZm4uNbO0Cn7tmm3G60JO8smGMu6U1CKpy91va3gTQzCz92ngaC8NTGL6qyJ7M7P7Jc3RwFVfOyTdKmmZpAcknSTpZUmXu3vDv3gr0dscDbx1/e/MzQc/Yze4t49JekLSOkn92eIFGvh8Xdhrl+hrngp43TjDDwiKM/yAoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwT1HyFe8ulrT1urAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADl5JREFUeJzt3X+wVPV5x/HPI7lABZwCilBCQkIgEUkC9Qac2jF0GC1WHSAZjDSTksbx2mlozNQ4Wv+o/uOMdqLG6VgmV0OCTjCSEpRknFTDtCVO9Mbrj+KPm4gaIsgNSKEBUX7ep3/cQ3vFe7677J7ds5fn/Zpxdvc858fj6uee3f2e3a+5uwDEc1rZDQAoB+EHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDUB5p5sOE2wkdqVDMPCYRyUAd02A9ZNevWFX4zWyjpbknDJN3n7rel1h+pUZpnC+o5JICELt9Y9bo1v+w3s2GS7pF0iaSZkpaZ2cxa9weguep5zz9X0qvu/rq7H5b0A0mLimkLQKPVE/7JkrYNeLw9W/YeZtZhZt1m1n1Eh+o4HIAi1RP+wT5UeN/3g929093b3b29TSPqOByAItUT/u2Spgx4/EFJO+prB0Cz1BP+pyVNN7OPmNlwSVdK2lBMWwAareahPnc/amYrJP2b+of6Vrn7S4V1BqCh6hrnd/dHJT1aUC8AmojLe4GgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiqrll6zWyrpP2Sjkk66u7tRTQFoPHqCn/mz9x9dwH7AdBEvOwHgqo3/C7pMTN7xsw6imgIQHPU+7L/AnffYWYTJD1uZr9y900DV8j+KHRI0kidXufhABSlrjO/u+/IbndJWi9p7iDrdLp7u7u3t2lEPYcDUKCaw29mo8xszPH7ki6W9GJRjQForHpe9p8tab2ZHd/PGnf/aSFdAWi4msPv7q9L+nSBvaBGw84cn1v79V0fSm47f/qWZP3Nzx5J1v3QoWQdrYuhPiAowg8ERfiBoAg/EBThB4Ii/EBQRXyrDw22a8WfJOs3X3t/bu3S0x+r69iLz7w8WT/65o669o/ycOYHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAY528Bw2ZMS9bvu+5byfrs4fn/Gftq6uj/9a4ck6xPumZisn6093d1doBG4cwPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzt8Cem4cm6x/aviwJnXyfl3nrUnWX3nycLL+uQf+Prf20VufS27bd/Bgso76cOYHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAqjvOb2SpJl0na5e6zsmXjJD0kaaqkrZKucPe9jWtzaBs2c0ay/rMF6e/rS3+QrN7+3+fk1rr/Jz1F90PTflrh2Gkz2oYn6/d+cWVu7fZVi5Lb9v3mtzX1hOpUc+b/nqSFJyy7UdJGd58uaWP2GMAQUjH87r5J0p4TFi+StDq7v1rS4oL7AtBgtb7nP9vdeyUpu51QXEsAmqHh1/abWYekDkkaqdMbfTgAVar1zL/TzCZJUna7K29Fd+9093Z3b2/TiBoPB6BotYZ/g6Tl2f3lkh4pph0AzVIx/Gb2oKQnJX3czLab2VWSbpN0kZltkXRR9hjAEFLxPb+7L8spLSi4l1PW7rnjk/WpH0h/FtKx7cJkffv5b+fWThv1TnLb8/7m75L1b1y9Nln/4pjcd3ySpAtH5td+vO6N5LYvX8qcAI3EFX5AUIQfCIrwA0ERfiAowg8ERfiBoPjp7iY4VuHCxj55sr75259M1sfpyfx9HziQ3HbSHb9I1tde/plkfdmYnyTr8vxJwnceSk//7QcPpfeNunDmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOdvgjGf761r+9//eXqsftx369p90j9+eEOFNWo/f/z8uU8k6zP2/rLmfaMyzvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/E2wf92k9ArnpstfntmVrG/6zNzc2ltzRie39ctOnIP1vWa1pcfae44cSdbPTUzhvf6Sf05ue8P5Vyfrempzuo4kzvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFTFcX4zWyXpMkm73H1WtuwWSVdLeitb7SZ3f7RRTQ51Ezf8Jll/5R8OJ+vXj385Wb/h4Z7cWqU5ASr5wmuXJuvvfu2sZH3Jg/+RW/vrM7Ylt33ta+lz07SnkmVUUM2Z/3uSFg6y/C53n539Q/CBIaZi+N19k6T0ZWAAhpx63vOvMLPNZrbKzMYW1hGApqg1/CslTZM0W1KvpDvyVjSzDjPrNrPuI2LuNaBV1BR+d9/p7sfcvU/SvZJyv1ni7p3u3u7u7W2qMGMlgKapKfxmNvBrakskvVhMOwCapZqhvgclzZd0ppltl3SzpPlmNluSS9oq6ZoG9gigAcy9vnHgk3GGjfN5tqBpxxsq3l46L1n/7jfvTNZntI3KrR3zvuS2H3ss/Z35T6z4VbLedyA9p8CWe/L/3bYsXpnc9uEDf5is37c0fQ1C33/lX/9wquryjdrne6yadbnCDwiK8ANBEX4gKMIPBEX4gaAIPxAUQ31DQKWhwD1XvJNbO/j79FWV51z/WrJ+bO/eZL2S08aMya29u258ctvHz12XrM/p+qtkffLnXkrWT0UM9QGoiPADQRF+ICjCDwRF+IGgCD8QFOEHgmKK7iFg9A/TU3SP/mHt+z5W+6ZV6du/P7e2b/2s9MYVpi6//VPp6wD+ZdL83NrR3t+ldx4AZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpxfpTmrG//Mlmfd8lfJutd561J1q/9xtTc2rTrGOfnzA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQVUc5zezKZLulzRRUp+kTne/28zGSXpI0lRJWyVd4e71/cg7YulL/5rA+DtOT9Z3P/Bust5z5T25tcvXpH/z35859X/zv5oz/1FJ17n7OZLOl/RVM5sp6UZJG919uqSN2WMAQ0TF8Lt7r7s/m93fL6lH0mRJiyStzlZbLWlxo5oEULyTes9vZlMlzZHUJelsd++V+v9ASJpQdHMAGqfq8JvZaEnrJH3d3fedxHYdZtZtZt1HdKiWHgE0QFXhN7M29Qf/++7+o2zxTjOblNUnSdo12Lbu3unu7e7e3qb0pJEAmqdi+M3MJH1HUo+73zmgtEHS8uz+ckmPFN8egEap5iu9F0j6kqQXzOz5bNlNkm6TtNbMrpL0hqSljWkRUZ32n88l6/NXX5+sv/yV/KG+/bemhwnPWJo/tbiU/knyoaJi+N39CUl5830vKLYdAM3CFX5AUIQfCIrwA0ERfiAowg8ERfiBoPjpbgxZH+vclqw/sHRibm3TJ/81ue3CT38lWT/tieeT9aGAMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4P4aso9u2J+trl3w2t/alnz2U3Hb39QeT9QlPJMtDAmd+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcX6cso71bMmtfeH1i5Pb/njOfcn6Vef/bfrgT21O11sAZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKriOL+ZTZF0v6SJkvokdbr73WZ2i6SrJb2VrXqTuz/aqEaBIr2zxJP1rl/8UbK+9+OjkvWxT510S01XzUU+RyVd5+7PmtkYSc+Y2eNZ7S53/2bj2gPQKBXD7+69knqz+/vNrEfS5EY3BqCxTuo9v5lNlTRHUle2aIWZbTazVWY2NmebDjPrNrPuIzpUV7MAilN1+M1stKR1kr7u7vskrZQ0TdJs9b8yuGOw7dy9093b3b29TSMKaBlAEaoKv5m1qT/433f3H0mSu+9092Pu3ifpXklzG9cmgKJVDL+ZmaTvSOpx9zsHLJ80YLUlkl4svj0AjWLu6SEPM/tTST+X9IL6h/ok6SZJy9T/kt8lbZV0TfbhYK4zbJzPswV1tgwgT5dv1D7fY9WsW82n/U9IGmxnjOkDQxhX+AFBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Kq+H3+Qg9m9pak3w5YdKak3U1r4OS0am+t2pdEb7UqsrcPu/tZ1azY1PC/7+Bm3e7eXloDCa3aW6v2JdFbrcrqjZf9QFCEHwiq7PB3lnz8lFbtrVX7kuitVqX0Vup7fgDlKfvMD6AkpYTfzBaa2a/N7FUzu7GMHvKY2VYze8HMnjez7pJ7WWVmu8zsxQHLxpnZ42a2JbsddJq0knq7xczezJ67583sL0rqbYqZ/buZ9ZjZS2Z2bba81Ocu0Vcpz1vTX/ab2TBJr0i6SNJ2SU9LWubuLze1kRxmtlVSu7uXPiZsZhdKelvS/e4+K1v2T5L2uPtt2R/Ose5+Q4v0doukt8ueuTmbUGbSwJmlJS2W9GWV+Nwl+rpCJTxvZZz550p61d1fd/fDkn4gaVEJfbQ8d98kac8JixdJWp3dX63+/3maLqe3luDuve7+bHZ/v6TjM0uX+twl+ipFGeGfLGnbgMfb1VpTfrukx8zsGTPrKLuZQZx9fGak7HZCyf2cqOLMzc10wszSLfPc1TLjddHKCP9gs/+00pDDBe7+x5IukfTV7OUtqlPVzM3NMsjM0i2h1hmvi1ZG+LdLmjLg8Qcl7Sihj0G5+47sdpek9Wq92Yd3Hp8kNbvdVXI//6eVZm4ebGZptcBz10ozXpcR/qclTTezj5jZcElXStpQQh/vY2ajsg9iZGajJF2s1pt9eIOk5dn95ZIeKbGX92iVmZvzZpZWyc9dq814XcpFPtlQxrckDZO0yt1vbXoTgzCzj6r/bC/1T2K6pszezOxBSfPV/62vnZJulvSwpLWSPiTpDUlL3b3pH7zl9DZfJzlzc4N6y5tZukslPndFznhdSD9c4QfExBV+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeC+l9zUxflH74X4gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADm5JREFUeJzt3XGQlPV9x/HP947zCAQRSjwJoBhDI9YpmJwYQ6alk8HRlAbtVCIzMTiT6aVtjLVjM7G0M3Ha6QzTaUxMm5peAhMyUZKMSiQTWmNpM5gxEg6bUfQErUEhUE6KCpjk4I5v/7gH54L3/HbZfXafPb7v1wxzu8/3+e3zndXPPbv3e3Z/5u4CEE9b2Q0AKAfhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1IRmHuwc6/SJmtzMQwKh/Epv6LgPWjX71hV+M7tW0j2S2iV9zd3XpPafqMm6yj5UzyEBJGzzLVXvW/PLfjNrl/RlSddJukzSSjO7rNbHA9Bc9bznXyTpBXd/0d2PS/qWpOXFtAWg0eoJ/yxJe0fd35dt+zVm1mNmfWbWd0KDdRwOQJHqCf9Yf1R4y+eD3b3X3bvdvbtDnXUcDkCR6gn/PklzRt2fLWl/fe0AaJZ6wr9d0jwzu9jMzpF0k6RNxbQFoNFqnupz9yEzu1XSIxqZ6lvn7s8U1hmAhqprnt/dN0vaXFAvAJqIy3uBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKqpS3QDo024oCtZPz7vnQ07dsfunyfru/7qXcn6ec+mV8Ge3v+rZL3tsf9O1puBMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBFXXPL+Z7ZF0VNKwpCF37y6iKYwfr3/s/cn6/304f777ziv+PTn24+c2bgHota9fmKz/4ZSNyfq0GyfWdfxls95X1/giFHGRz++5+6ECHgdAE/GyHwiq3vC7pB+Y2Q4z6ymiIQDNUe/L/sXuvt/Mzpf0qJk95+5bR++Q/VLokaSJmlTn4QAUpa4zv7vvz34OSNooadEY+/S6e7e7d3eos57DAShQzeE3s8lmNuXUbUnXSNpZVGMAGquel/1dkjaa2anHud/d03M3AFpGzeF39xclLSiwFzRA24L5yfpzn56crD92zReT9Xe0b08fv0UnlD4x9eUKe9Q3jz8etOZ/GQANR/iBoAg/EBThB4Ii/EBQhB8Iiq/uPsu9cfGUZH33dfdWeIS3FddMk33ltfyv377vpSub2MlbTdULpR5f4swPhEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exz98EE2bPStb7Pzs7We96PL0c9LkbnsittQ16cuzuE8eT9b1D5yXrcya8lqzfsnNVbu3V/t9Iju3anu79vMf3Jut+7Fhubepr5c+zl40zPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExTx/AdrPm5qsL/r+z5L1787YlKwv7rv1jHs6pfPf0l+t/ZnfvyVZH35mV7LePn9esj591//k107uTo6tZKiu0eDMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBVZznN7N1kpZJGnD3y7Nt0yV9W9JcSXskrXD3VxvXZvnaJuYv2Tz4QHqef/WM/0zW3/PQnyXrl258JlkfTlbTKs3jVxzf/3xd41Geas78X5d07Wnb7pS0xd3nSdqS3QcwjlQMv7tvlXT4tM3LJa3Pbq+XdH3BfQFosFrf83e5+wFJyn6eX1xLAJqh4df2m1mPpB5JmqhJjT4cgCrVeuY/aGYzJSn7OZC3o7v3unu3u3d3qLPGwwEoWq3h3yTp1NeyrpL0cDHtAGiWiuE3sw2SfizpPWa2z8w+IWmNpKVm9rykpdl9AONIxff87r4yp/ShgnspVfu0acn6c3/3m7m1XfP/JTl2x2D62Jf+7YvJ+vCRI+kHAGrAFX5AUIQfCIrwA0ERfiAowg8ERfiBoPjq7sz+j81P1nfd8E+5tU1vpKcJ1y5bmqwPv5L/9dZAo3DmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgmOfPHL3qlzWPvedn6U83v2038/hoPZz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAo5vkzGxb3Vtgj//fkA5d9Mzny6rvvSNYv3nQ8WW//4ZPJOlALzvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFTFeX4zWydpmaQBd78823aXpD+W9Eq222p339yoJpthUWdHsn7Ch3Nr09omJsc+99Evpx97Rf5jS9LlW/4kWZ+6Pf/4x2Z7cuy56dXBNeOpN9I7VHDotyfn1rp+OJAcO8z3IDRUNWf+r0u6doztX3D3hdm/cR18IKKK4Xf3rZION6EXAE1Uz3v+W83sKTNbZ2bp9aoAtJxaw3+vpEskLZR0QNLn83Y0sx4z6zOzvhMarPFwAIpWU/jd/aC7D7v7SUlflbQosW+vu3e7e3eHOmvtE0DBagq/mc0cdfcGSTuLaQdAs1Qz1bdB0hJJM8xsn6TPSVpiZgsluaQ9kj7ZwB4BNIC5p+eBi3SuTferLP0d92XZ/a9XpuvLvtKkTuL4yaAl67c/e1OyPn3Z7iLbOSts8y064ofTT2yGK/yAoAg/EBThB4Ii/EBQhB8IivADQTHVl7EJ6Useji9ZkFv7+D9/Lzl2Ulv6suZlk15J1jusPVk/W53UyWT9t+6/LVm/5DM/LrKdcYGpPgAVEX4gKMIPBEX4gaAIPxAU4QeCIvxAUCzRnfGhoWS94z925NY2XPrOuo79pT9Kf3R1uCM9bfuBv/xJbm3NBdtr6qkVtFU4N81ecKBJnZydOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDM87eAyQ9sq2v89xZcnVtbc3N6nv8XfjxZf9/WP03WL/pa+rsGDt32i9xa35XfTI5FY3HmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgKs7zm9kcSd+QdIGkk5J63f0eM5su6duS5kraI2mFu7/auFaR58JHEusC3JweO8nOSdb7f3dtsn7zRUuT9c1zH0lU6zv3vPy/05P1edpT1+Of7ap59ock3eHu8yW9X9KnzOwySXdK2uLu8yRtye4DGCcqht/dD7j7k9nto5L6Jc2StFzS+my39ZKub1STAIp3Rq+7zGyupCskbZPU5e4HpJFfEJLOL7o5AI1TdfjN7O2SHpR0u7sfOYNxPWbWZ2Z9J5Resw5A81QVfjPr0Ejw73P3h7LNB81sZlafKWlgrLHu3uvu3e7e3aHOInoGUICK4Tczk7RWUr+73z2qtEnSquz2KkkPF98egEapuES3mX1Q0mOSnpbeXDN5tUbe939H0oWSXpZ0o7sfTj1WKy/RPZ61TZmSWxu4f2Zy7BPv3VB0O1Ub9BPJ+rJn019pPmlFemZ5+LXXz7in8e5MluiuOM/v7j+SlPdgJBkYp7jCDwiK8ANBEX4gKMIPBEX4gaAIPxAUX919Fjh59Ghu7YJPT0uO/YN1H0nWV8/9frJ+dedwsv7gsRm5tb/e/NHk2Hf/xRPJevrIqIQzPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EVfHz/EXi8/zjz8HbPpCsH73yl8n6pX9zKLc29NLemnpCvjP5PD9nfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8Iis/zI6nrS4+n6xXGDxXXCgrGmR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqoYfjObY2b/ZWb9ZvaMmf15tv0uM/u5mf00+/fhxrcLoCjVXOQzJOkOd3/SzKZI2mFmj2a1L7j7PzauPQCNUjH87n5A0oHs9lEz65c0q9GNAWisM3rPb2ZzJV0haVu26VYze8rM1pnZmOtCmVmPmfWZWd8JDdbVLIDiVB1+M3u7pAcl3e7uRyTdK+kSSQs18srg82ONc/ded+929+4OdRbQMoAiVBV+M+vQSPDvc/eHJMndD7r7sLuflPRVSYsa1yaAolXz136TtFZSv7vfPWr7zFG73SBpZ/HtAWiUav7av1jSzZKeNrOfZttWS1ppZgsluaQ9kj7ZkA4BNEQ1f+3/kaSxvgd8c/HtAGgWrvADgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EZe7evIOZvSLppVGbZkg61LQGzkyr9taqfUn0Vqsie7vI3d9RzY5NDf9bDm7W5+7dpTWQ0Kq9tWpfEr3VqqzeeNkPBEX4gaDKDn9vycdPadXeWrUvid5qVUpvpb7nB1Cess/8AEpSSvjN7Foz22VmL5jZnWX0kMfM9pjZ09nKw30l97LOzAbMbOeobdPN7FEzez77OeYyaSX11hIrNydWli71uWu1Fa+b/rLfzNol7Za0VNI+SdslrXT3Z5vaSA4z2yOp291LnxM2s9+RdEzSN9z98mzbP0g67O5rsl+c09z9sy3S212SjpW9cnO2oMzM0StLS7pe0i0q8blL9LVCJTxvZZz5F0l6wd1fdPfjkr4laXkJfbQ8d98q6fBpm5dLWp/dXq+R/3maLqe3luDuB9z9yez2UUmnVpYu9blL9FWKMsI/S9LeUff3qbWW/HZJPzCzHWbWU3YzY+jKlk0/tXz6+SX3c7qKKzc302krS7fMc1fLitdFKyP8Y63+00pTDovd/b2SrpP0qezlLapT1crNzTLGytItodYVr4tWRvj3SZoz6v5sSftL6GNM7r4/+zkgaaNab/Xhg6cWSc1+DpTcz5taaeXmsVaWVgs8d6204nUZ4d8uaZ6ZXWxm50i6SdKmEvp4CzObnP0hRmY2WdI1ar3VhzdJWpXdXiXp4RJ7+TWtsnJz3srSKvm5a7UVr0u5yCebyviipHZJ69z975vexBjM7F0aOdtLI4uY3l9mb2a2QdISjXzq66Ckz0n6rqTvSLpQ0suSbnT3pv/hLae3JRp56frmys2n3mM3ubcPSnpM0tOSTmabV2vk/XVpz12ir5Uq4XnjCj8gKK7wA4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1P8Df2wQFZ1eHhYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAADupJREFUeJzt3X+QVfV5x/HPw7KArDgRLT+KRCwihtGR1C2akmnoODIm4wix0ZFmUjJ13HQCbeiYNJZpJ6TTH5Qk/piMcWZNaDBRY2YMSidMIqG2ak0IK6GCwUZLVkQoq0WUGH7t8vSPPZtZYc/3Xu6vc3ef92uG2XvPc86ex+t+9ty733PO19xdAOIZVXQDAIpB+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBDW6kTsbY2N9nNoauUsglKN6R8f9mJWzblXhN7PrJN0jqUXS1919dWr9cWrTVXZNNbsEkLDFN5e9bsVv+82sRdK9kj4saY6kJWY2p9LvB6CxqvnMP0/Sy+6+292PS/qOpEW1aQtAvVUT/mmSXh30fG+27F3MrMPMusys64SOVbE7ALVUTfiH+qPCadcHu3unu7e7e3urxlaxOwC1VE3490qaPuj5BZL2VdcOgEapJvxbJc0ys4vMbIykWyRtqE1bAOqt4qE+d+81s+WSfqj+ob617v5CzToDUFdVjfO7+0ZJG2vUC4AG4vReICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqpqll4z65Z0WFKfpF53b69FUwDqr6rwZ/7Q3d+owfcB0EC87QeCqjb8LukJM3vOzDpq0RCAxqj2bf98d99nZpMkbTKzF939qcErZL8UOiRpnMZXuTsAtVLVkd/d92VfeyStlzRviHU63b3d3dtbNbaa3QGooYrDb2ZtZjZh4LGkhZJ21qoxAPVVzdv+yZLWm9nA93nI3X9Qk64A1F3F4Xf33ZKuqGEvABqIoT4gKMIPBEX4gaAIPxAU4QeCIvxAULW4qg/D2Ki5c5L1o1PakvXuxZasf2ze1tzaCW9Jbvvkt047YfRdpv7HW8m6/+yFZD06jvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/COAz5+bW9u9LL3tQx+4P1m/ckx6LL6uPvfTZPnIZ48n652H8s9h+Np/fSi57axbdyXrJ48eTdaHA478QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/xN4OQH88fpJan70+ntvz//3tzazNFnldh7ehx/05H09it/vjhZP7TnPbm1nYu/mtz2bw9cnayvmdKVrF9x1iu5tTvnPZLc9q//8pPJ+gX/9GyyPhxw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMzd0yuYrZV0vaQed78sWzZR0iOSZkjqlnSzu79Zamfn2ES/yq6psuXhZ/dD6XH8B+t4Tf2SX16brG998aJk/dLPlLiu/Z13zrinAZN/fE6y3vMXFybrl9z3YrL+N5P/Pbf29JGpyW1vaEv/OC++elGy3vvq3mS9Xrb4Zr3tB9OTKWTKOfJ/U9J1pyy7Q9Jmd58laXP2HMAwUjL87v6UpIOnLF4kaV32eJ2k9GleAJpOpZ/5J7v7fknKvk6qXUsAGqHu5/abWYekDkkap/H13h2AMlV65D9gZlMlKfvak7eiu3e6e7u7t7dqbIW7A1BrlYZ/g6Sl2eOlkh6vTTsAGqVk+M3sYUk/ljTbzPaa2a2SVku61sxeknRt9hzAMFLyM7+7L8kphRqwH9WWP0/9S393eXLbXR/Kv95ekkaVuKZ+67H0uRgffzz/5vyzv5gep7/kUPqa+JPJanUun/Basr5pdPochK4vXZmsn3fnltza4rZDyW2lsobKhzXO8AOCIvxAUIQfCIrwA0ERfiAowg8Exa27y3TohvzhvH+76cvJbUeVOK1585H0mY+rP700Wb/4iZ/k1vqSW1bPRqd/hEbNnplb+/pjE5PbfumBdcn65WNyTyzN5L/uLZY+7l2+5Y+T9Wk9/1Ni382PIz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4f5k8cdXtUa/u8s/DJ9PTYP/vVWOS9SM3zsutXTxrf0U9DXjr6Lhk/aYLtyXry97zrdxa1/H0f9f8saUuKK78tnD/eTT9vaf9ffr/qR87VvG+mwVHfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquQU3bU0nKfoHjVhQm7tyKPnJbf99qXfTtYnt6TH+VstfWvvPq/8BtvHvDdZH2vNeypIb4m7FSx4/pbc2sRl6W17d3dX0lLhaj1FN4ARiPADQRF+ICjCDwRF+IGgCD8QFOEHgio5iGtmayVdL6nH3S/Llq2SdJuk17PVVrr7xno12QxOHj6cWxu7ML8mSR2Tb0zWd62akawvvHJHsv6Ltybl1l557fzkti1j0uPdN8x+PllfMyU9xXc9zXmyI1mffXv+FOC9B0rd83/kK+fI/01J1w2x/C53n5v9G9HBB0aikuF396ckHWxALwAaqJrP/MvN7HkzW2tm59asIwANUWn475M0U9JcSfslfSVvRTPrMLMuM+s6oeF/3zNgpKgo/O5+wN373P2kpPsl5d5B0t073b3d3dtblZ6QEkDjVBR+M5s66OlHJe2sTTsAGqWcob6HJS2QdL6Z7ZX0BUkLzGyuJJfULelTdewRQB1wPT+S9q2fk6xvn5e+V0FKd++vk/XFX/2rZH3a3T9N1r03fa+CkYjr+QGURPiBoAg/EBThB4Ii/EBQhB8Iqnnvy4yG+OU/fiBZ3/Z7d5X4DulptlM+tiY9lPfb9z6brDdukHpk4sgPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzj/C7fvc7yfrP/z4mmT9LBtf1f7vefPi3NqUf9me3LbyicdRDo78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/wjwImF7bm1x5anx/HfO7q6cfw9JW6/veHz+bdqH/vrrVXtG9XhyA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQZUc5zez6ZIekDRF/ZdYd7r7PWY2UdIjkmZI6pZ0s7u/Wb9Wkaf7+pbc2owqx/H396XH8f9kxe3J+vjvb6lq/6ifco78vZJud/f3Sbpa0jIzmyPpDkmb3X2WpM3ZcwDDRMnwu/t+d9+WPT4saZekaZIWSVqXrbZO0uJ6NQmg9s7oM7+ZzZD0fklbJE129/1S/y8ISZNq3RyA+ik7/GZ2tqRHJa1w97fPYLsOM+sys64TOlZJjwDqoKzwm1mr+oP/oLt/L1t8wMymZvWpknqG2tbdO9293d3bWzW2Fj0DqIGS4Tczk/QNSbvc/c5BpQ2SlmaPl0p6vPbtAaiXci7pnS/pE5J2mNnAvZZXSlot6btmdqukPZJuqk+LaDlvYrL+sxvvTlSre7e14JnlyfrM9QzlDVclw+/uz0iynHL+xdoAmhpn+AFBEX4gKMIPBEX4gaAIPxAU4QeC4tbdTaDl3HOT9RVbnk7Wz7bKx/L/+f/el6zPuu2lZJ1ptIcvjvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/E3gjRsuTdYXjn8yWe/zyve98YsLkvW2d7hef6TiyA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHO3wT+6LM/Stb7vPKr5i/+1z9L1i95lHH8qDjyA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQJcf5zWy6pAckTVH/bdo73f0eM1sl6TZJr2errnT3jfVqdCS74qw9yXqLpX9H/+RoX25tzpqe5La9ySpGsnJO8umVdLu7bzOzCZKeM7NNWe0ud/9y/doDUC8lw+/u+yXtzx4fNrNdkqbVuzEA9XVGn/nNbIak90saOCd0uZk9b2ZrzWzIOafMrMPMusys64SOVdUsgNopO/xmdrakRyWtcPe3Jd0naaakuep/Z/CVobZz9053b3f39lZVPqccgNoqK/xm1qr+4D/o7t+TJHc/4O597n5S0v2S5tWvTQC1VjL8ZmaSviFpl7vfOWj51EGrfVTSztq3B6Beyvlr/3xJn5C0w8y2Z8tWSlpiZnMluaRuSZ+qS4cBrHjw1mT9xdu+lqz/6do/z61N3/1sRT1h5Cvnr/3PSLIhSozpA8MYZ/gBQRF+ICjCDwRF+IGgCD8QFOEHgjL3KuZ3PkPn2ES/yq5p2P6AaLb4Zr3tB4camj8NR34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKqh4/xm9rqkVwYtOl/SGw1r4Mw0a2/N2pdEb5WqZW8XuvtvlbNiQ8N/2s7Nuty9vbAGEpq1t2btS6K3ShXVG2/7gaAIPxBU0eHvLHj/Kc3aW7P2JdFbpQrprdDP/ACKU/SRH0BBCgm/mV1nZv9tZi+b2R1F9JDHzLrNbIeZbTezroJ7WWtmPWa2c9CyiWa2ycxeyr4OOU1aQb2tMrPXstduu5l9pKDeppvZk2a2y8xeMLPPZMsLfe0SfRXyujX8bb+ZtUj6haRrJe2VtFXSEnf/eUMbyWFm3ZLa3b3wMWEz+wNJv5L0gLtfli1bI+mgu6/OfnGe6+6fb5LeVkn6VdEzN2cTykwdPLO0pMWSPqkCX7tEXzergNetiCP/PEkvu/tudz8u6TuSFhXQR9Nz96ckHTxl8SJJ67LH69T/w9NwOb01BXff7+7bsseHJQ3MLF3oa5foqxBFhH+apFcHPd+r5pry2yU9YWbPmVlH0c0MYXI2bfrA9OmTCu7nVCVnbm6kU2aWbprXrpIZr2utiPAPdYuhZhpymO/uvyvpw5KWZW9vUZ6yZm5ulCFmlm4Klc54XWtFhH+vpOmDnl8gaV8BfQzJ3fdlX3skrVfzzT58YGCS1OxrT8H9/EYzzdw81MzSaoLXrplmvC4i/FslzTKzi8xsjKRbJG0ooI/TmFlb9ocYmVmbpIVqvtmHN0hamj1eKunxAnt5l2aZuTlvZmkV/No124zXhZzkkw1l3C2pRdJad/+HhjcxBDP7HfUf7aX+SUwfKrI3M3tY0gL1X/V1QNIXJD0m6buS3itpj6Sb3L3hf3jL6W2B+t+6/mbm5oHP2A3u7YOSnpa0Q9LJbPFK9X++Luy1S/S1RAW8bpzhBwTFGX5AUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4L6f8MXKrwouVOnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction 9\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "\n",
    "\n",
    "# 通过tf.layers来定义模型结构。这里可以使用原生态TensorFlow API或者任何TensorFlow\n",
    "# 的高层封装。x给出了输入层张量，is_training指明了是否为训练。该函数返回前向传播的结果。\n",
    "def lenet(x, is_training):\n",
    "    x = tf.reshape(x, shape=[-1, 28, 28, 1])\n",
    "    conv1 = tf.layers.conv2d(x, 32, 5, activation=tf.nn.relu)\n",
    "    conv1 = tf.layers.max_pooling2d(conv1, 2, 2)\n",
    "    conv2 = tf.layers.conv2d(conv1, 64, 3, activation=tf.nn.relu)\n",
    "    conv2 = tf.layers.max_pooling2d(conv2, 2, 2)\n",
    "    fc1 = tf.contrib.layers.flatten(conv2)\n",
    "    fc1 = tf.layers.dense(fc1, 1024)\n",
    "    fc1 = tf.layers.dropout(fc1, rate=0.4, training=is_training)\n",
    "    return tf.layers.dense(fc1, 10)\n",
    "\n",
    "\n",
    "# 自定义Estimator中使用的模型。定义的函数有4 个输入：\n",
    "#    features给出了在输入函数中会提供的输入层张量。注意这是一个字典，字典里的内容是\n",
    "#       通过tf.estimator.inputs.numpy_input_fn中x参数的内容指定的。\n",
    "#    labels是正确答案，这个字段的内容是通过numpy_input_fn 中y参数给出的。\n",
    "#    mode 的取值有3 种可能，分别对应Estimator类的train 、evaluate和predict这3 个函数。\n",
    "#       通过这个参数可以判断当前是有是训练过程。\n",
    "#    params是一个字典，这个字典中可以给出模型相关的任何超参数。比如这里的学习率。\n",
    "def model_fn(features, labels, mode, params):\n",
    "    # 定义神经网络的结构并通过输入得到前向传播的结果\n",
    "    predict = lenet(\n",
    "        features[\"image\"], mode == tf.estimator.ModeKeys.TRAIN)\n",
    "\n",
    "    # 如果在预测模式，那么只需要将前向传播结果返回\n",
    "    if mode == tf.estimator.ModeKeys.PREDICT:\n",
    "        return tf.estimator.EstimatorSpec(\n",
    "            mode=mode,\n",
    "            predictions={\"result\": tf.argmax(predict, 1)})\n",
    "\n",
    "    # 定义损失函数\n",
    "    loss = tf.reduce_mean(\n",
    "        tf.nn.sparse_softmax_cross_entropy_with_logits(\n",
    "           logits=predict, labels=labels))\n",
    "\n",
    "    # 定义优化函数\n",
    "    optimizer = tf.train.GradientDescentOptimizer(\n",
    "        learning_rate=params[\"learning_rate\"])\n",
    "\n",
    "    # 定义训练过程\n",
    "    train_op = optimizer.minimize(\n",
    "        loss=loss, global_step=tf.train.get_global_step())\n",
    "\n",
    "    # 定义评价标准\n",
    "    eval_metric_ops = {\n",
    "        \"accuracy\": tf.metrics.accuracy(\n",
    "            tf.argmax(predict, 1), labels)\n",
    "    }\n",
    "\n",
    "    # 返回模型训练过程需要使用的损失函数、训练过程和评测方法\n",
    "    return tf.estimator.EstimatorSpec(\n",
    "        mode=mode,\n",
    "        loss=loss,\n",
    "        train_op=train_op,\n",
    "        eval_metric_ops=eval_metric_ops)\n",
    "\n",
    "\n",
    "# 建立Estimator\n",
    "model_params = {\"learning_rate\": 0.01}\n",
    "estimator = tf.estimator.Estimator(model_fn=model_fn, \n",
    "                                   params=model_params,\n",
    "                                   model_dir='log')\n",
    "\n",
    "# 加载数据\n",
    "mnist = input_data.read_data_sets(\"../../datasets/MNIST_data\", one_hot=False)\n",
    "\n",
    "# 训练\n",
    "train_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "      x={\"image\": mnist.train.images},\n",
    "      y=mnist.train.labels.astype(np.int32),\n",
    "      num_epochs=None,\n",
    "      batch_size=128,\n",
    "      shuffle=True)\n",
    "\n",
    "estimator.train(input_fn=train_input_fn, steps=10000)\n",
    "\n",
    "# 测试\n",
    "test_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "      x={\"image\": mnist.test.images},\n",
    "      y=mnist.test.labels.astype(np.int32),\n",
    "      num_epochs=1,\n",
    "      batch_size=128,\n",
    "      shuffle=False)\n",
    "\n",
    "test_results = estimator.evaluate(input_fn=test_input_fn)\n",
    "print(\"\\ntest_results: \", test_results, '\\n')\n",
    "\n",
    "# 预测\n",
    "predict_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "      x={\"image\": mnist.test.images[:10]},\n",
    "      num_epochs=1,\n",
    "      shuffle=False)\n",
    "\n",
    "predictions = estimator.predict(input_fn=predict_input_fn)\n",
    "for i, p in enumerate(predictions):\n",
    "    plt.imshow(mnist.test.images[i].reshape([28, 28]))\n",
    "    plt.show()\n",
    "    print(\"Prediction %s\" % p[\"result\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出，**Estimator能非常好地支持自定义模型，而且模型结构的定义过程中也可以使用其他的TensorFlow高层封装（比如代码中使用到的tf.layers）。Estimator在支持自定义模型结构的同时，并不影响它对训练过程的封装。**\n",
    "\n",
    "### 10.3.3 使用数据集（datasets）作为Estimator输入\n",
    "为了更加方便地介绍数据集（Dataset）和Estimator的结合方法，本节将使用[iris分类数据集](http://archive.ics.uci.edu/ml/datasets/Iris)。iris数据集需要通过**4个特征来分辨3种类型的植物**。iris 数据集中总共包含了**150个样本，其中包括120条训练数据、30条测试数据**。这些数据都存储在csv 文件中。以下代码介绍了如何通过Estimator 和数据集相结合的方式完成整个数据读取和模型训练的过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "WARNING:tensorflow:Using temporary folder as model directory: C:\\Users\\MSI-PC\\AppData\\Local\\Temp\\tmpahgtht_g\n",
      "INFO:tensorflow:Using config: {'_model_dir': 'C:\\\\Users\\\\MSI-PC\\\\AppData\\\\Local\\\\Temp\\\\tmpahgtht_g', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001E8A3D0CC88>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into C:\\Users\\MSI-PC\\AppData\\Local\\Temp\\tmpahgtht_g\\model.ckpt.\n",
      "INFO:tensorflow:loss = 41.302673, step = 0\n",
      "INFO:tensorflow:global_step/sec: 319.028\n",
      "INFO:tensorflow:loss = 9.177642, step = 100 (0.313 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.606\n",
      "INFO:tensorflow:loss = 4.334622, step = 200 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.561\n",
      "INFO:tensorflow:loss = 2.868094, step = 300 (0.272 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 375 into C:\\Users\\MSI-PC\\AppData\\Local\\Temp\\tmpahgtht_g\\model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 3.6491613.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-11-21-11:50:52\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from C:\\Users\\MSI-PC\\AppData\\Local\\Temp\\tmpahgtht_g\\model.ckpt-375\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-11-21-11:50:52\n",
      "INFO:tensorflow:Saving dict for global step 375: accuracy = 0.96666664, average_loss = 0.08816624, global_step = 375, loss = 2.644987\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 375: C:\\Users\\MSI-PC\\AppData\\Local\\Temp\\tmpahgtht_g\\model.ckpt-375\n",
      "\n",
      "Test accuracy: 96.6667 %\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "\n",
    "# Estimator的自定义输入函数需要每一次被调用时可以得到一个batch的数据（包括所有的\n",
    "# 输入层数据和期待的正确答案标注），通过数据集可以很自然地实现这个过程。虽然Estimator\n",
    "# 要求的自定义输入函数不能有参数，但是通过pythoy提供的lambda表达式可以快速将下面的\n",
    "# 函数转化为不带参数的函数。\n",
    "def my_input_fn(file_path, perform_shuffle=False, repeat_count=1):\n",
    "    # 解析csv文件中一行\n",
    "    def decode_csv(line):\n",
    "        parsed_line = tf.decode_csv(line, [[0.], [0.], [0.], [0.], [0]])\n",
    "        # Estimator的输入函数要求特征是一个字典，所以这里返回的也需要是一个字典。\n",
    "        # 字典中key的定义需要和DNNClassifier中feature_columns的定义匹配：x。\n",
    "        return {\"x\": parsed_line[:-1]}, parsed_line[-1:]\n",
    "\n",
    "    # 使用数据集输入数据。具体使用方法参考第7章。\n",
    "    dataset = tf.data.TextLineDataset(file_path).skip(1).map(decode_csv)\n",
    "    if perform_shuffle:\n",
    "        dataset = dataset.shuffle(buffer_size=256)\n",
    "    dataset = dataset.repeat(repeat_count) \n",
    "    dataset = dataset.batch(32) \n",
    "    iterator = dataset.make_one_shot_iterator()\n",
    "    # 通过定义的数据集得到一个batch的输入数据。这个就是整个自定义的输入过程的返回结果\n",
    "    batch_features, batch_labels = iterator.get_next()\n",
    "    return batch_features, batch_labels\n",
    "\n",
    "# 与10.3.1小节中类似地定义Estimator\n",
    "feature_columns = [tf.feature_column.numeric_column(\"x\", shape=[4])]\n",
    "classifier = tf.estimator.DNNClassifier(\n",
    "    feature_columns=feature_columns,  \n",
    "    hidden_units=[10, 10],  \n",
    "    n_classes=3)\n",
    "\n",
    "# 使用lambda表达式将训练的相关信息传入自定义输入数据处理函数，并生成需要的输入函数\n",
    "classifier.train(\n",
    "    input_fn=lambda: my_input_fn(\"../../datasets/iris_training.csv\", True, 100))\n",
    "\n",
    "# 对于测试，同样使用lambda，这样可以大大减少冗余代码\n",
    "test_results = classifier.evaluate(\n",
    "    input_fn=lambda: my_input_fn(\"../../datasets/iris_test.csv\", False, 1))\n",
    "print(\"\\nTest accuracy: %g %%\" % (test_results[\"accuracy\"]*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过以上代码，可以看出Estimator可以非常好地和数据集结合，这样就能够很容易地支持海量数据读入或者复杂的数据预处理流程。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
